From eab14f74c881bd0f3687659cf4c0c10f1805e6eb Mon Sep 17 00:00:00 2001 From: Ram Damera Date: Thu, 14 Feb 2019 21:00:06 +0530 Subject: [PATCH 1/2] Doc changes --- README.md | 159 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 94 insertions(+), 65 deletions(-) diff --git a/README.md b/README.md index d4698c9..e6eb2ec 100644 --- a/README.md +++ b/README.md @@ -1,76 +1,97 @@ -__Scripts to copy [Rocks Clusters](http://www.rocksclusters.org) roll isos to a generic (linux) webserver for faster local hosting__ - -Purpose: Rocks 7.0 frontends require a web server to have rolls "installed" to download. Sometimes distance or -other issues indicate that a local copy on an existing (non-Rocks) web-server would be much better. The scripts in this -repository help with that. - -There are four scripts defined in this roll -* rollcopy.sh - copy a roll iso contents for web serving -* index.cgi - cgi to present roll directory in a standard format -* httpdconf.sh (optional) - defines a virtual server httpd configuration file -* unpack-guides (optional) - unpacks roll users guides into a web-servable directory - -__0. Preparation__ -* clone this repository `git clone https://github.com/rocksclusters/roll-server.git` -* download the roll isos from UCSD. - -__I. Copy Roll(s) to a web directory__ -For this example, will assume the `/var/www/html/rocks/7.0` is the (already created) parent directory into -which you want to copy rolls. The `rollcopy.sh` script will create the subdirectory `install/rolls`. -to copy the iso file, `CentOS-7.4.1708-0.x86_64.disk1.iso` to the the release directory: -``` -# rollcopy.sh CentOS-7.4.1708-0.x86_64.disk1.iso /var/www/html/rocks/7.0 -``` -Repeat the above for each roll iso that you want to copy +# roll-server + +Scripts to help hosting a custom [Rocks Clusters](http://www.rocksclusters.org) roll download +server using apache/httpd + +Rocks 7.0 frontend installation downloads "rolls" from remote server and installs them. The minimum +rolls required would end up 10 GB or above and downloading them for every installation is hard. +The scripts in this repository can help with that. We setup a local server that can simulate the +rocks cluster download server for easy download. -__II. Make certain that the supplied `index.cgi` script gives the directory listing__ +There are four scripts defined in this repo +* `index.cgi` - cgi script to serve roll directory in a standard format +* `rollcopy.sh` - extract roll iso contents for web serving +* `httpdconf.sh` (optional) - defines a virtual server httpd configuration file +* `unpack-guides` (optional) - unpacks roll users guides into a web-servable directory + +## 0. Preparation +* Clone this repository `git clone https://github.com/rocksclusters/roll-server.git` +* Download the roll isos from [rockscluster.org](http://central-7-0-x86-64.rocksclusters.org/isos/) + +## I. Extract roll(s) to a web directory +`rollcopy.sh` extracts the roll iso files. The first argument `` is the iso roll and the +second one is the server directory. It creates `install/rolls/` directory in the target and extracts +the files to it. + +``` sh +$ rollcopy.sh ``` -# cp index.cgi /var/www/html/rocks/7.0/install/rolls + +To extract the roll, `CentOS-7.4.1708-0.x86_64.disk1.iso` to the the server directory: +``` sh +$ rollcopy.sh CentOS-7.4.1708-0.x86_64.disk1.iso /var/www/html/ ``` -You will probably need to add something like the following in your Apache -configuration, so that the directory index is given via the index.cgi +Repeat the above for each roll iso that you want to copy. The rolls will be accessbile for download +from the path `http://localhost/install/rolls/`. + +## II. Copy `index.cgi` script for directory listing +Rocks frontend installation program searches for all the rolls by checking the directory listing of +files from the server. The `index.cgi` script servers the response by mimicking the rolls download +server. It needs to be copied to every directory. + +``` sh +$ find /var/www/html/install/rolls -type d -exec cp index.cgi {} \; ``` + +Now configure `httpd` to execute the cgi script for every request + +``` apache # allow all access to the rolls RPMS - - AddHandler cgi-script .cgi - Options FollowSymLinks Indexes ExecCGI - DirectoryIndex /rocks/7.0/install/rolls/index.cgi + + # don't use SetHandler + AddHandler cgi-script .cgi + Options +FollowSymLinks +Indexes +ExecCGI + DirectoryIndex /install/rolls/index.cgi Allow from all ``` (See below for an example of how to test and verify that your index.cgi is being called properly) -__III. (Optional) provide a virtual httpd server__ -The script `httpdconf.sh` will write to standard output a reasonable stanza for Apache web server configuration. -Suppose that you wanted the virtual server to be called `rocks-7.my.org` and have it used the directory above. -Then -``` -httpdconf.sh rocks-7-0.my.org /var/www/html/rocks/7.0 -``` -will generate an httpds conf file. It is your responsibility to place this on the appropriate directory and restart -your webserver. +## III. (Optional) serve directories on a virtual server +`httpdconf.sh` lets you generate a vhost configuration for the server -__IV. (Optional) unpack the userguides__ -You can unpack userguides -``` -unpack-guides.sh /var/www/html/rocks/7.0/install/rolls /var/www/html/rocks/7.0 +``` sh +$ httpdconf.sh rocks-7-0.my.org /var/www/html/ ``` -__V. Verifying that your output is correct (that is, index.cgi is being called)__ -try +Above command generates a httpd vhost conf file with cgi handlers for virtual server +`rocks-7.0.my.org`. If you have your rolls in `/var/www/html/rocks/7.0/[install/rolls/...]` then +change the second argument accordingly. + +## IV. (Optional) unpack the userguides +You can unpack userguides +``` sh +$ unpack-guides.sh /var/www/html/rocks/7.0/install/rolls/ /var/www/html/rocks/7.0/ ``` -wget -O - http://central-7-0-x86-64.rocksclusters.org/install/rolls + +## V. Verifying that cgi scripts are working + +Try +``` sh +$ wget -O - http://central-7-0-x86-64.rocksclusters.org/install/rolls/ ``` + and compare the output of the rocks central server listing of rolls to -your website, something like -``` -wget -O - http://rocks-7-0.my.org/install/rolls +your website, something like +``` sh +$ wget -O - http://rocks-7-0.my.org/install/rolls/ ``` -You should see output very similar to + +You should see output very similar to ``` TTP request sent, awaiting response... 200 OK Length: 877 [text/html] @@ -124,23 +145,31 @@ Saving to: ‘STDOUT’ zfs-linux/ -100%[======================================>] 877 --.-K/s in 0s +100%[======================================>] 877 --.-K/s in 0s ``` -__VI. Verifying that rolls directories can be listed__ + +## VI. Verifying that rolls directories can be listed The rolls themselves need be listed. Try the following + +``` sh +$ wget -O - http://rocks-7-0.my.org/install/rolls/base/7.0/x86_64 ``` -wget -O - http://central-7-0-x86-64.rocksclusters.org/install/rolls/base/7.0/x86_64 -``` -and verify that when you use your roll server instead that you get the same output +and verify that you get the same output from +`http://central-7-0-x86-64.rocksclusters.org/install/rolls/base/7.0/x86_64` -__VII. Other items to check__ +## VII. Other items to check The following may need to be checked/changed for your setup: - -* Firewall needs modifying on your web server to allow http access (platform sepcific) - -* SELinux may need to be modified or turned off (e.g., ```setenforce Permissive```) - - - + * Firewall needs modifying on your web server to allow http access (platform sepcific) + * SELinux may need to be modified or turned off (e.g., ```setenforce Permissive```) + * Check your permissions on the directories, run + ``` sh + $ chown www-data:www-data -R /var/www/html/rocks/7.0/ + ``` + to fix permissions to apache user. (run with sudo if necessary) + * You might need root pemissions to run `rollcopy.sh` + * Need to enable `cgi` module in apache/httpd + ``` sh + $ a2enmod cgi + ``` From a4ec3635b26809ce0a116f1ba5154961be7096d3 Mon Sep 17 00:00:00 2001 From: Ram Damera Date: Sun, 17 Feb 2019 15:34:01 +0530 Subject: [PATCH 2/2] Check for gzip --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index e6eb2ec..fe80ed0 100644 --- a/README.md +++ b/README.md @@ -173,3 +173,4 @@ The following may need to be checked/changed for your setup: ``` sh $ a2enmod cgi ``` + * Disable `gzip` compression