Behrooz File System (BFS) Wiki

Welcome to the BFS wiki!

BFS is a distributed, scalable, replicated in-memory filesystem. BFS provides a POSIX compatible interface and can be mounted under any directory and be used same as a traditional filesystem such as Ext4 or NTFS.

Installation

BFS can be used in three different modes:

1. STANDALONE

In this mode BFS is only an in-memory filesystem and data are not backed anywhere; therefore, data kept in BFS are not persistent and after unmount there is no trace of your data.

2. STANDALONE_SWIFT

This mode is similar to STANDALONE mode; however, data are backed on a swift storage as well. In order to use this mode, it is necessary to provide a working configuration for Swift backend storage.

3. DISTRIBUTED

In this mode all the features of BFS are available. Different nodes in a cluster will have the same transparent access to the BFS. Data are distributed among nodes and are accessed through network.

In order to compile BFS, you need to have the following dependencies in your system available:

• GCC,G++ >= 4.9
• POCO C++ libraries >= 1.6
• Swift_C++_SDK
• FUSE >= 2.9.3
• ZooKeeper >= 3.4.6
• PF_RING
• ntp for clock synchronization

In the following, a brief explanation is provided for installing and configuring each of the required tools for BFS.

First you need to install common build tools which usually exist in every development machine. These packages can be installed using the following command in a debian based distribution:

 sudo aptitude install gcc-4.9 g++-4.9 make automake autoconf build-essential htop libcap2-bin

Note: it is necessary to have gcc and g++ >= 4.9 because BFS uses some features of C++ which are only include in g++ 4.9.

In order to patch fuse module, you will need kernel headers as well:

sudo apt-get install linux-headers-$(uname -r)

#POCO POCO libraries can be downloaded and installed directly form POCO website:

 wget http://pocoproject.org/releases/poco-1.6.0/poco-1.6.0.tar.gz
 tar -xzvf poco*
 cd poco*
 ./configure --config=Linux --omit=Data/ODBC,Data/MySQL,Zip,NetSSL_OpenSSL,Crypto  --no-tests --no-samples --no-wstring
 make all -j 4
 sudo make install

#fuse Fuse 2.9.3 can be downloaded and installed using the following commands:

wget http://iweb.dl.sourceforge.net/project/fuse/fuse-2.X/2.9.3/fuse-2.9.3.tar.gz; tar -xzvf fuse*
cd fuse*
./configure
make -j 4 
sudo make install

this will install default fuse module in kernel we need to modify it because vanilla fuse does not support block sizes larger than 128KB. Note that the following trick works only if your kernel is configured to have fuse and a module (-m) not statically linked (-y). You can check your kernel config file using the following command:

  cat /boot/config-`uname -r`

Look for FUSE. If fuse is statically linked in your kernel then you need to recompile the whole kernel once and specify fuse as a module. Modifying fuse kernel Module:

 sudo aptitude install linux-source-`uname -r`
 cd /usr/src
 tar xf linux-source-`uname -r`
 cd linux-source-`uname -r`/fs/fuse

Modify FUSE_MAX_PAGES_PER_REQ to 16M which is pages 4096 pages(4kpage) in fuse_i.h file. Rebuild the module:

 sudo make -C /lib/modules/`uname -r`/build M=$PWD

then you need to replace existing fuse module(you can use "modinfo fuse" to see where it's

 sudo cp fuse.ko /lib/modules/`uname -r`/kernel/fs/fuse/
 sudo cp cuse.ko /lib/modules/`uname -r`/kernel/fs/fuse/

Finally, you need a restart.

#Swift CPP SDK

git clone https://github.com/bshafiee/Swift_CPP_SDK.git
cd Swif*
make all -j 4
sudo make install

#Zookeeper

wget http://mirror.csclub.uwaterloo.ca/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
tar -xzvf zooke*
cd zoo*
cd src/c
./configure;make all -j 4
sudo make install
sudo ldconfig

Remember after running zookeeper server for the first time you need to create /BFSElection node. You can do this by issuing 'create /BFSElection ""' in the zookeeper client.

#PFRING

sudo aptitude install subversion libnuma-dev
svn co https://svn.ntop.org/svn/ntop/trunk/PF_RING/
cd PF_RING/kernel
./configure
make all -j 4
sudo make install
cd ../userland/lib
./configure --disable-bpf
make all -j 4
sudo make install

You can load pf_ring module using the following command and configuration:

sudo modprobe pf_ring enable_tx_capture=0 quick_mode=1 min_num_slots=8192

and then check if it is loaded:

sudo lsmod|grep pf_ring

#NTP In distributed mode it's beneficial to synchronize nodes using ntp.

sudo aptitude install ntp

Type ntpq -p to see servers you are syncing with. Type date again to see if the time changed. You time should be synced in a next minute. Done

#BFS By this point you should have met all the dependencies of BFS. You can Download and compile BFS using the following commands:

git clone https://github.com/bshafiee/BFS.git
cd BFS
make all -j 4

#BFS Configuration file: Configuration of BFS are kept in "config" file:

#Mode determines whether to operate in DISTRIBUTED, STANDALONE_SWIFT or STANDALONE mode 
mode = STANDALONE
#Swift Authentication info
username = admin
password = behrooz
auth_url = http://10.42.0.83:5000/v2.0/tokens
tenant_name = admin
#what percentage of system memory to use? (0,1]
max_mem_coef = 0.7;
#zero networking device
network_dev=wlan0
#tcp port
tcp_port=5555
#Zookeeper configuration
zoo_server_url=192.168.249.101:2181,10.42.0.97:2181,10.42.0.62:2182,129.97.170.232:2181,10.42.0.1:2181
zoo_election_znode=/BFSElection
zoo_assignment_znode=/BFSTasks
#Debug SWIFT_CPP_SDK
debug_swift_cpp_sdk = false

#Running BFS You can run BFS by running mount.sh script: "./mount.sh" in mount.sh script the last argument to BFS is the mount directory which is mountdir/ by default. You can change it to your desired destination.