Using the CHMI Linux server
πŸ‘©πŸ½β€πŸ’»

Using the CHMI Linux server

About our server

The CHMI server is a Linux machine running Ubuntu 20.04. It has 512Gb of RAM, a 2Tb internal hard drive, and two 6-core Intel Xeon E5-2643v4 CPUs (total of 12 CPU cores). Additional hard drive storage is accessible through a hot-swap bay in the front of the computer and includes an 8Tb drive (mounted as /publicData) and an 8Tb RAID1 arrayed hard drive (mounted as /data) – in total 18Tb of storage are available. Software available on this machine changes from time to time, but you can always view a relatively up-to-date listing of informatics tools we have installed here. In addition to this software, we also host RStudio Server Pro on this machine, which allows anyone with an account on the server to run RStudio in their browser with full access to this machine's compute capabilities. We also have Geneious Prime installed and accessible remotely through a desktop sharing protocol.

General rules

Use of the server is free-of-charge to approved PennVet labs. However, to protect the machine from abuse and prevent issues with storage, we have a few rules all users must follow. Failure to adhere to these rules will result in a warning. Additional violations will result in suspension of your account. Here are the rules:

  • We will set-up your account and install any software you may need. Please do not install software yourself unless you’ve cleared it is first.
  • In addition to your home account which is located at /home/username on the server, you will also be given a data folder that is located on /data/username. All data must be stored in this directory not your /home/username folder.
  • Data storage space is limited (18Tb goes quickly!), so we periodically monitor storage space and may request that data not currently in use be moved off the server.
  • We require a strong password for each account. We will set this password for you, and ask that you do not change it.
  • CHMI is not responsible for backing up your raw data or analysis scripts and files.

Connect to CHMI linux server via SSH

πŸ“Œ

Accounts on our Linux server will only be granted to labs at PennVet. Contact Dan Beiting if you're at PennVet and would like to request an account.

πŸ“Œ

The steps outlined below require you to work in your terminal, bash, or command prompt program (exact name depends on your operating system). We have found that the terminal available through RStudio provides the most stable connection with few, if any, broken connections.

In Sept 2020, UPenn choseOh to prohibit all SSH connections through its firewall. As a result, you will need to first use UPenn's GlobalProtect VPN client. Once connected to the VPN (requires your PennKey and password), then you will be able to use SSH to connect to our server (requires your server username and password). Here's how you can connect to the VPN:

  • Open your browser and navigate to UPenn's VPN client. You'll be prompted to download GlobalProtect for your computer (you'll only need to do this once).
  • Once installed and launched, you should see a small globe icon in your toolbar. Click the globe icon, paste vpn.upenn.edu and click 'connect'.
  • You'll be directed back to your browser to authenticate with your PennKey and password. Once connected to the VPN, you can SSH to our server as indicated below.
# connect by entering the line below in your terminal or bash program.  Replace 'username' with your username
ssh username@130.91.255.137

Transfer files to/from our linux server via SFTP

As above with SSH, this will only work if you are connected to UPenn's VPN. There are many ways to transfer files between your local computer and our linux server. We recommend using a file transfer protocol (FTP) client, and find that the FileZilla Client is a good choice because it is free and easy to use.

image

Connecting to the PennVet Storage server from our Linux

Our Linux server is for computing, not for storage. It is the responsibility of each user (that's you!) to make sure you have your data securely backed-up.

We strongly recommend that you use the PennVet Storage Server, which provides archival tape-based backup of data for PennVet labs. If your lab does not have an account on the storage server, please contact your PI and have them work with PennVet IT support to get this set up. Once set-up you can easily connect to the PennVet Storage Server from our linux.

To move data from our Linux server to the storage server you will either need to be sitting at our Linux, or you will need to contact Dan Beiting to gain access to the Linux via our remote desktop connection service.

Once connected to (or sitting in front of) our Linux, follow the 4 steps shown in the screenshot below to connect to smb://fileserver.vet.upenn.edu/groups$/

image

Once you've connected, you'll be asked to enter your credentials and click connect again. See screenshot below:

image

If you entered the correct credentials, you will see the PennVet Storage Server connection listed on the left-hand side of your file browser. Click on this and you'll see your lab folder(s) displayed (screenshot below). You should now be able to drag files from our Linux to your lab's storage! Transfer time can be lengthy and will depend on how much data you're trying to move.

image

Understanding Linux permissions

If you're trying to do something to a file or folder on the linux (e.g., delete, move, edit, create) and are unable to, chances are you need to modify the permissions (chmod), the owner (chown) or the group (chgrp). See below and read more about permissions here

image

Helpful command line tips

If you’re new to the command-line in Linux, there are lots of online resoures for learning, but here are a few of the commands that will help you move around and carry out basic tasks. Note that many of these may only work if run as sudo.

Also, check out this github page for a really awesome list of modern unix tools (each tool requires download and installation if you're not working on our Linux). In particular, dust, lsd and broot are all awesome.

Common bash commands

typing this at the promptdoes this
tar -xvzf [fileName.tar.gz]
unzip a .tar file
standard would be ls and ls -l, but lsd and lsd -l is way better πŸ™‚
list all files and folders in your working diretory with info on permissions
standard would be du -a -h --max-depth=1 | sort -hr, but dust is way better πŸ™‚
lists all files and folders in your working directory sorted by size
standard would be du -sh *, but dust is way better πŸ™‚
simpler version of the command above. lists all files in a folder and shows their file size
standard would be df -h, but duf is way better πŸ™‚
view free/used disk space by drive
ls -l | wc -l
counts files in a directory
tree -d, but broot is way better πŸ™‚
lists all files and folders in your working directory as a tree structure
lsblk
lists drives and their size (as well as used/free space on each)
pressing up arrow
recalls previous command
cd /
takes you to the root directory
cd ~
takes you to your home directory
cd ..
takes you up one level in your file directory
cd ../..
takes you up two levels in your file directory
chmod u+x [fileName]
edits permissions on file
chown [yourUserName] [fileName]
makes you the owner of a file
chgrp [yourUserName] [fileName]
assigns you as the group for the file
rm -rf [directoryName]
removes a folder and all of its contents
wget [URLtoFile]
downloads a file from a website
defaults write com.apple.finder AppleShowAllFiles YES
show all hidden files in the finder (Mac only)
pip freeze
lists all the python packages (and their versions) installed on the server
sudo nano /etc/profile
opens up the system profile where new program paths can be added to the system PATH
export PATH="/path/to/your/software/:$PATH"
add a new piece of software to the system PATH so it is executable from anywhere
alias something="something else"
add lines like this to your ~/.bash_profile to create a keyboard shortcut, in this case typing 'something' actually does 'something else'