Skip to content

imp1 #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Jan 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
53f23d7
fix: specify absolute path on cp nginx.conf
mickamy Dec 31, 2023
5543dc6
docker: name app image
mickamy Dec 31, 2023
a5e2dfa
chore: add note for first bench
mickamy Jan 1, 2024
4c02e3c
chore: TZ/LANG 設定 6098
mickamy Jan 1, 2024
eb3ff5c
ruby: update to 3.1.4 6948
mickamy Jan 1, 2024
f901141
candidates をメモリに載せる 7654
mickamy Jan 1, 2024
56e4534
Revert "candidates をメモリに載せる 7654"
mickamy Jan 2, 2024
fb6b2d1
bundle update and specify ruby version 7144
mickamy Jan 2, 2024
85bb5e5
increase unicorn worker and make candidates on memory (but the change…
mickamy Jan 2, 2024
f511f9e
chore: install htop
mickamy Jan 2, 2024
6510847
workload と unicorn worker を増やす workload: 4, unicorn: 12 16942
mickamy Jan 2, 2024
c23c349
ALTER TABLE votes ADD INDEX idx_votes_candidate_id (candidate_id);
mickamy Jan 2, 2024
5d88879
unicorn: 24 多分意味ない 19568
mickamy Jan 2, 2024
7d3c233
chore: fix ci
mickamy Jan 2, 2024
7e78231
unicorn: 16, workload: 6 24094
mickamy Jan 2, 2024
79e8d82
Merge branch 'mickamy' into imp1
mickamy Jan 2, 2024
2650939
chore: fix CI
mickamy Jan 2, 2024
8a6b65f
ruby: remove pid file before running unicorn
mickamy Jan 2, 2024
4e0e4fe
candidates をメモリに載せる 21250
mickamy Jan 2, 2024
14b879c
chore: fix run script
mickamy Jan 2, 2024
dc039e6
chore: fix run_app
mickamy Jan 2, 2024
b3cd226
fix: do not exit on mysql service not started on first time
mickamy Jan 2, 2024
1053d21
votes に count column を追加して、投票数を保存 27926
mickamy Jan 2, 2024
0f75144
add foreign key to votes to candidates 28840
mickamy Jan 2, 2024
6952689
election_results の複数回コールを止める 33168
mickamy Jan 2, 2024
0373d6a
ALTER TABLE votes ADD INDEX idx_votes_candidate_and_count (candidate_…
mickamy Jan 2, 2024
9732ad2
YJIT 有効化(なぜかローカルだと遅くなったけど速くなるはずなので CI で確認) 19963
mickamy Jan 2, 2024
1ac1fce
静的ファイルが nginx で返せていないようなので revisit(なんだかローカルが遅い) 22143
mickamy Jan 2, 2024
90fe4a9
nginx のログを捨てる
mickamy Jan 2, 2024
c3501cd
最新の ruby を使用(3.2.2)
mickamy Jan 2, 2024
34883eb
keyword をテーブルにする 27440
mickamy Jan 2, 2024
ae26de0
add desc to count index 29668
mickamy Jan 2, 2024
b5127fd
set content max age to /, /candidates/:id and /political_parties/:nam…
mickamy Jan 2, 2024
36a4397
cache at nginx level
mickamy Jan 2, 2024
e56f3c0
workload を 8 に 18968
mickamy Jan 2, 2024
649c4a4
more cache on nginx
mickamy Jan 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ down:
docker compose down;

bench:
docker exec -i ishocon2-bench-1 sh -c "./benchmark --ip app:443";
docker exec -i ishocon2-bench-1 sh -c "./benchmark --ip app:443 --workload 6"

bench-with-db-init: up
docker exec -i ishocon2-bench-1 sh -c " \
service mysql restart \
&& tar -jxvf /root/admin/ishocon2.dump.tar.bz2 && mysql -u root -pishocon ishocon2 < /root/admin/ishocon2.dump \
&& ./benchmark --ip app:443 \
&& ./benchmark --ip app:443 --workload 8 \
";

change-lang:
Expand Down
7 changes: 7 additions & 0 deletions admin/config/bashrc
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
export TZ=Asia/Tokyo

export PATH="$HOME/.rbenv/bin:$PATH"
if command -v rbenv 1>/dev/null 2>&1; then
eval "$(rbenv init -)"
fi
export PATH="$HOME/.rbenv/shims:$PATH"

# 美しくないが bundle が見つからないと言われるので仕方ない
# export PATH="$HOME/.gem/ruby/3.1.4/bin:$PATH"

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
Expand All @@ -27,3 +32,5 @@ alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'
alias grep='grep --color=auto'

export ISHOCON_APP_LANG=ruby
42 changes: 40 additions & 2 deletions admin/config/nginx.conf
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
user www-data;
worker_processes 4;
worker_processes 16;
pid /run/nginx.pid;

events {
Expand All @@ -10,19 +10,57 @@ http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

access_log /var/log/nginx/access.log;
# access_log /var/log/nginx/access.log;
access_log off;

error_log /var/log/nginx/error.log;

upstream app {
server 127.0.0.1:8080;
}

# cache の設定
proxy_cache_path /var/nginx/cache_1 levels=1:2 keys_zone=political_parties_cache:1m max_size=1g inactive=1m use_temp_path=on;
proxy_cache_path var/nginx/cache_2 levels=1:2 keys_zone=candidates_cache:1m max_size=1g inactive=1m use_temp_path=on;
proxy_cache_path var/nginx/cache_3 levels=1:2 keys_zone=index_cache:1m max_size=1g inactive=1m use_temp_path=on;

server {
listen 443;
ssl on;
ssl_certificate /etc/nginx/ssl/server.crt;
ssl_certificate_key /etc/nginx/ssl/server.key;

rewrite ^/(.*)/$ /$1 permanent;

location /css/ {
add_header Cache-Control "s-maxage=86400";
alias /home/ishocon/public/css/;
}

location /political_parties/ {
proxy_cache political_parties_cache;
proxy_cache_valid 200 302 30s;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_set_header Host $host;
proxy_pass http://app;
}

location /candidates/ {
proxy_cache candidates_cache;
proxy_cache_valid 200 302 30s;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_set_header Host $host;
proxy_pass http://app;
}

location = / {
proxy_cache index_cache;
proxy_cache_valid 200 302 30s;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_set_header Host $host;
proxy_pass http://app;
}

location / {
proxy_set_header Host $host;
proxy_pass http://app;
Expand Down
5 changes: 3 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,15 @@ services:
interval: 10s
timeout: 3s
retries: 5
start_period: 10s
start_period: 30s
command: [/home/ishocon/run.sh]
tty: true
volumes:
- storage_app:/var/lib/mysql
# - storage_app:/var/lib/mysql
- ./webapp:/home/ishocon/webapp
ports:
- "443:443"
- "3306:3306"

bench:
image: showwin/ishocon2_bench:latest
Expand Down
5 changes: 4 additions & 1 deletion docker/app/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
#!/bin/bash -eux

sudo mkdir -p /usr/share/nginx/var/nginx/cache_1 /usr/share/nginx/var/nginx/cache_2 /usr/share/nginx/var/nginx/cache_3 /usr/share/nginx/var/nginx/cache_4 /usr/share/nginx/var/nginx/cache_5
sudo mkdir -p /var/nginx/cache_1 /var/nginx/cache_2 /var/nginx/cache_3 /var/nginx/cache_4 /var/nginx/cache_5
sudo nginx -t
sudo service nginx start
sudo service mysql start
sudo service mysql start || true
sudo chown -R mysql:mysql /var/lib/mysql /var/run/mysqld
sudo service mysql start # 正しく起動
sudo mysql -u root -pishocon -e 'CREATE DATABASE IF NOT EXISTS ishocon2;' && \
Expand Down
32 changes: 28 additions & 4 deletions docker/app/ruby/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,23 +1,47 @@
FROM showwin/ishocon2_app_base:latest
ENV APP_LANG 'Ruby'

# Ruby のインストール
# TZ の設定
ENV TZ Asia/Tokyo

RUN sudo apt-get update
RUN sudo apt-get install -y ruby-dev libmysqlclient-dev && \

# 日本語の設定
RUN sudo apt-get install -y language-pack-ja htop && \
sudo update-locale LANG=ja_JP.UTF-8 && \
sudo rm -rf /var/lib/apt/lists/*
ENV LANG="ja_JP.UTF-8" \
LANGUAGE="ja_JP:ja" \
LC_ALL="ja_JP.UTF-8"

# Ruby のインストール
RUN sudo apt-get update && \
sudo apt-get install -y ruby-dev libmysqlclient-dev libffi6 libffi-dev libyaml-dev && \
git clone https://github.com/sstephenson/rbenv.git ~/.rbenv
RUN PATH="$HOME/.rbenv/bin:$PATH" && \
eval "$(rbenv init -)" && \
git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build && \
rbenv install 2.5.1 && rbenv rehash && rbenv global 2.5.1
rbenv install 3.2.2 && rbenv rehash && rbenv global 3.2.2

# YJIT の有効化
ENV RUBY_YJIT_ENABLE=1

# アプリケーション
RUN mkdir /home/ishocon/data /home/ishocon/webapp
COPY admin/ishocon2.dump.tar.bz2 /home/ishocon/data/ishocon2.dump.tar.bz2
COPY webapp/ /home/ishocon/webapp
COPY admin/config/bashrc /home/ishocon/.bashrc

# 静的ファイルの配置(nginx が読めるように)
COPY --chown=www-data:www-data public /home/ishocon/public

# base をビルドし直しても変更されなかったのでここでも改めてコピーする
COPY admin/config/nginx.conf /etc/nginx/nginx.conf

# ライブラリのインストール
RUN cd /home/ishocon/webapp/ruby && sudo gem install bundler -v "1.16.1" && bundle install
RUN cd /home/ishocon/webapp/ruby && \
. /home/ishocon/.bashrc && \
bash ./set_up_ruby.sh

WORKDIR /home/ishocon
EXPOSE 443
Expand Down
15 changes: 14 additions & 1 deletion docker/benchmarker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,17 @@ FROM ubuntu:18.04
RUN sudo apt-get update
RUN apt-get install -y wget

# TZ の設定
ENV TZ Asia/Tokyo

# 日本語の設定
RUN sudo apt-get install -y language-pack-ja && \
sudo update-locale LANG=ja_JP.UTF-8 && \
sudo rm -rf /var/lib/apt/lists/*
ENV LANG="ja_JP.UTF-8" \
LANGUAGE="ja_JP:ja" \
LC_ALL="ja_JP.UTF-8"

# Go のインストール
RUN wget -q https://dl.google.com/go/go1.13.15.linux-amd64.tar.gz && \
tar -C /usr/local -xzf go1.13.15.linux-amd64.tar.gz
Expand All @@ -14,7 +25,9 @@ ENV PATH $PATH:$GOROOT/bin
# MySQL のインストール
RUN ["/bin/bash", "-c", "debconf-set-selections <<< 'mysql-server mysql-server/root_password password ishocon'"]
RUN ["/bin/bash", "-c", "debconf-set-selections <<< 'mysql-service mysql-server/mysql-apt-config string 4'"]
RUN apt-get install -y mysql-server

RUN sudo apt-get update && \
apt-get install -y mysql-server

COPY admin/ /root/admin/

Expand Down
Loading