Lustre with Virtualbox install: Difference between revisions
(An up to date installation guide for Lustre using VirtualBox and Rocky Linux) |
mNo edit summary |
||
Line 47: | Line 47: | ||
https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-modules-internal-4.18.0-513.18.1.el8_lustre.x86_64.rpm | https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-modules-internal-4.18.0-513.18.1.el8_lustre.x86_64.rpm | ||
https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/p7zip-16.02-20.el8.x86_64.rpm | https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/p7zip-16.02-20.el8.x86_64.rpm | ||
https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/q/quilt-0.66-2.el8.noarch.rpm</pre> | https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/q/quilt-0.66-2.el8.noarch.rpm'</pre> | ||
Now that we have them all downloaded we can install them all at once using: | Now that we have them all downloaded we can install them all at once using: | ||
<pre>sudo dnf install *</pre> | <pre>sudo dnf install *</pre> |
Revision as of 12:03, 21 May 2024
Introduction
This page provides an up to date installation guide for Lustre, using VirtualBox and Rocky Linux.
Building Lustre with LDISKFS (as of 2024-05-20)
The following guide uses VirtualBox 7.0, Rocky Linux 8.9 and Lustre 2.15.63 on a 64-bit operating system.
Setting up VirtualBox
Previous knowledge of VirtualBox is helpful, but not required for this tutorial. As previously stated we will be using VirtualBox 7.0 and Rocky Linux 8.9. We recommend the following minimum hardware settings for the VM:
- 4-8gb ram
- 4 cpu cores
- 30gb hard drive space
After installing the Rocky Linux ISO we will change the network settings on virtual box to enable SSH forwarding. In VirtualBox go to settings -> network and make sure it is attached to NAT. Then click on Port Forwarding and add a new rule called "SSH forwarding". The protocol should be TCP, and for this tutorial we will set the host port to 1122 but can be set to anything. Set the guest port to 22 and leave the Host IP/Guest IP empty.
Setting up Rocky Linux
By default Rocky Linux does not have it's internet connection enabled. If you have installed the GUI version of Rocky simply go to settings and enable the connection. Also go to the advanced settings and check off "enable by default" to avoid having to do this step after every reboot. However if you are using the minimal install ISO the quickest way to get it up and running is to use the
sudo nmuti
command. Enable the wired connection. Edit the connection afterwards to enable the automatically connect option as well. While in the nmtui menu we will also set the hostname to "Lustre-Server" for clarity. After you have set the hostname reboot the VM and we are now ready to begin install Lustre.
Installing Lustre (Client and Server)
Installing dependencies
Now for the part we've all been waiting for. First things first we can finally ssh into our VM from our host computer. Replace "yourname" with whatever username you created while making your VM.
ssh -p 1122 [email protected]
After logging in we will create a few directories to keep things organized.
mkdir Lustre cd Lustre mkdir lustre-server-packages cd lustre-server-packages
We will now install various packages and dependencies required to build and compile Lustre.
sudo dnf install git libtool flex bison wget sudo dnf --enablerepo=devel install libmount-devel libyaml-devel libnl3-devel e2fsprogs-devel
We will now install a few specific kernel packages required for LDISKFS using wget
wget -i - <<< 'https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-core-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-debuginfo-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-debuginfo-common-x86_64-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-devel-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-headers-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-modules-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://build.whamcloud.com/job/lustre-master/arch=x86_64,build_type=server,distro=el8.9,ib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/RPMS/x86_64/kernel-modules-internal-4.18.0-513.18.1.el8_lustre.x86_64.rpm https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/p7zip-16.02-20.el8.x86_64.rpm https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/q/quilt-0.66-2.el8.noarch.rpm'
Now that we have them all downloaded we can install them all at once using:
sudo dnf install *
The last change we need to make is to patch the e2fsprogs to the Lustre version. This can be done by editing the /etc/dnf/dnf.conf file and adding
[Lustre-e2fsprogs] name=Lustre-e2fsprogs baseurl=http://downloads.whamcloud.com/public/e2fsprogs/latest/el$releasever/ gpgcheck=0 enabled=1
After saving the changes to the dnf.conf file we can apply the patch.
sudo dnf update e2fsprogs
Everything should now be installed, we can now compile Lustre
Compiling and Building Lustre
We can now begin the process of compiling our Lustre server and client. Before downloading Lustre from git make sure you go back to the original /Lustre folder that we made. The below code does this for you if you haven't already.
cd .. git clone git://git.whamcloud.com/fs/lustre-release.git cd lustre-release
With the repository downloaded we can begin the build by running these commands:
sh autogen.sh ./configure sudo make sudo make install
The make process might take a while, this is normal. Reboot the VM one last time after the make install command is finished. After the reboot we have successfully installed Lustre.
Testing and Build confirmation
To ensure that what we have done was successful we can go into the tests folder and run the llmount.sh script to mount the LDISKFS
cd /Lustre/lustre-release/lustre/tests sudo ./llmount.sh
After running this script we should be able to see them mounted in /mnt
cd /mnt ls
If you can see "lustre lustre-mds1 lustre-ost1 lustre-ost2" everything has been built properly. You can now also run the rest of the tests that are included in the lustre/tests folder.