1. Create an account: First things first, sign up and create and account. You can use your normal Amazon account, but it'll require special info for AWS. They'll ask for you credit card information and phone number which will be used for verification.
2. Create an Instance: Once you've created the account goto the AWS Management Console and then the Amazon EC2 tab. On the left within the Navigation bar will be "EC2 Dashboard", go there and click the "Launch Instance" button under getting started on the right.
2a. AMI: At the time of this writing the most applicable AMI is "Cluster Instances Amazon Linux AMI 2011.02.1 (AMI Id: ami-321eed5b)" . That should be under the "Quick Start" tab. Hit the select button next to it. 2b. Instance Type: For GPU cracking, you'll need to select the Instance type of "Cluster GPU (cg1.4xlarge)" from the drop down. Once selected, Click the continue button. You can click the continue button again on the next page ("Advanced Instance Properties"). 2c. Tags: The most basic tag is the Name and the tag is already created for you. Give it a name of something useful by putting something in the Value column in the Name row. Click the continue button. 2d. Key Pair: The keypair will be used for your SSH session into the box. If you've already created a keypair for a previously created instance you can reuse it. Otherwise create a new one. Just name it something useful (doesnt have to be instance specific). Click "Create & Download your Key Pair". It'll prompt you to download your .pem file. Do it and then the wizard will redirect you to the next screen (Security Groups). Take the default value and click "Continue". At this point it may ask you to create firewall rules if you havent already. The default option is good enough for this set up which will allow SSH inbound.
3. Connect to your Instance: After your instance is created, head on over to "Instances" on the left navigation bar under the Amazon EC2 tab. Select your newly created instance under the "My Instances" section, and within the lower pane, scroll down to get the "Public DNS" name. This is what you'll use as the hostname to connect to.
3a. Preparing the SSH key and connecting (Linux): if you're using Linux things a pretty straight forward:
root@bt:~# chmod 400 key.pem
root@bt:~# ssh -i key.pem ec2-user@blah.compute-1.amazonaws.com
- 3b. Preparing the SSH key and connecting (Windows): if you're using Windows with putty, you'll have to format the key to use it. Use puttygen.
- Import the key (File - Load Private Key, View all files, then select your key.pem). You'll get a success message on import.
- If you want to set a passphrase on it, set one with the "Key passphrase" and "Confirm passphrase" fields
- Click the "Save Private Key" button. it'll prompt you if you didnt set a passphrase (you dont have to, click "Yes" at this warning if you didnt).
- Define a save location and name. We'll use "key.ppk" for our example.
- Now open up putty, set your hostname in the hostname field.
- Expand "SSH" in the Category pane on the left then goto "Auth".
- Click browse, select your "key.ppk" and click "Open" in the browse window, and then again in the putty window to connect to the instance.
- When prompted enter username: ec2-user
[ec2-user@aws ~]$ cat setup_aws_gpu.sh
#!/bin/bash
# Pyrit Install (Amazon GPU Instance)
# originally from http://borrellstudios.com/2011/06/pyrit-on-amazon-ec2/
# modified by brad a
echo "[+] Installing packages"
# added a couple more packages to support crunch
sudo yum -qy install subversion python-devel openssl-devel zlib-devel libpcap-devel glibc-devel gcc
echo "[+] Fetching Scapy"
wget http://www.secdev.org/projects/scapy/files/scapy-latest.zip
echo "[+] Unziping Scapy"
unzip scapy-latest.zip -d scapy
echo "[+] Installing Scapy"
cd scapy/*/
sudo python setup.py install
cd ../../
echo "[+] Fetching pyrit"
svn checkout http://pyrit.googlecode.com/svn/trunk/ pyrit_svn
echo "[+] Building pyrit"
cd pyrit_svn/pyrit
python setup.py build
echo "[+] Installing pyrit"
sudo python setup.py install
echo "[+] Moving onto cpyrit_cuda"
cd ../cpyrit_cuda
echo "[+] Fixing setup.py"
sed -i -e "s/NVIDIA_INC_DIRS = \[\]/NVIDIA_INC_DIRS = \[\'\/opt\/nvidia\/cuda\/include\'\]/" setup.py
echo "[+] Building cpyrit"
python setup.py build
echo "[+] Installing cpyrit"
sudo python setup.py install
echo "NOTE: FOR DISTERIBUTED PROCESSING:"
echo -e "\tOpen TCP port 17935 on all machines"
echo -e "\tOn the master server, add the IP addresses of the slaves in .pyrit/config"
echo -e "\tSet rpc_server = true on all machines"
echo ""
echo -e "\tStart the slaves with pryit serve"
echo -e "\tRun pryit benchmark on the master server to see if it all works"
So just go ahead and run that script:
[ec2-user@aws ~]$ chmod +x setup_aws_gpu.sh
[ec2-user@aws ~]$ ./setup_aws_gpu.sh
and it should do it all for you. Afterwards test to make sure it worked with:
[ec2-user@aws ~]$ pyrit list_cores
Hello, facing problem execute above command.
ReplyDeleteFailed to execute command 'nvcc -V'
Traceback (most recent call last):
File "setup.py", line 175, in
setup(**setup_args)
File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/usr/lib64/python2.6/distutils/command/build.py", line 134, in run
self.run_command(cmd_name)
File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "setup.py", line 82, in run
raise SystemError("Nvidia's CUDA-compiler 'nvcc' can't be " \
SystemError: Nvidia's CUDA-compiler 'nvcc' can't be found.
If you're still having trouble, make sure you selected the correct Amazon Machine Image on EC2. I got this to run straight out of the linked script using
DeleteEC2 CentOS 5.5 GPU HVM AMI (Driver 260.19.29) - ami-42a2532b
and with the above instructions using
amzn-ami-gpu-hvm-2013.09.2.x86_64-ebs - ami-4ba18d22
These are COMMUNITY AMIs, not base Quick Start AMIs.
Cannot detect the 2 x NVIDIA Tesla “Fermi” M2050 GPUs in the list :(
ReplyDeleteI confirm that does not work. the problem is to install pyrit cuda .. the solution?
ReplyDeleteIf you're still having trouble, make sure you selected the correct Amazon Machine Image on EC2. I got this to run straight out of the linked script using
ReplyDeleteEC2 CentOS 5.5 GPU HVM AMI (Driver 260.19.29) - ami-42a2532b
and with the above instructions using
amzn-ami-gpu-hvm-2013.09.2.x86_64-ebs - ami-4ba18d22
These are COMMUNITY AMIs, not base Quick Start AMIs.