Software Development Blog

Exploration of Big Data, Machine Learning, Nature Language Processing, and other fun projects!

Setup Ethereum Mining on Ubuntu in Azure - Part 2

Setup Ethereum Mining on Ubuntu in Azure - Part 2

Part 2 (this article) will specifically detail how to:

  1. Install GPU drivers for your Ubuntu instance with a NVIDIA M60 or K80 GPU cards.
  2. Install and run an Ethereum miner (called ethminer) using your GPU and send your profits to your account address.

(My address is 0xB82adE3cEEE35afcF3415afE983d3E14218A41c3, and Ethereum donations can be sent there if you enjoy this article.)

First, spin up an Azure instance with Ubuntu 16. When you pick the instance size, be sure to pick either the NV6 or NC6 sizes that have a GPU attached. (You could pick the larger NV or NC Azure instances too with multiple GPUs attached if you want.)

Warning: Be sure to use Ubuntu 16 and not Ubuntu 14 as the drivers are different.

 

1) After your Ubuntu 16 Azure NC or NV instance is created, login with PuTTY and verify your GPU is available by running this command:

lspci

2) Next, run these update commands:

sudo apt-get update

sudo apt-get upgrade -y

sudo apt-get dist-upgrade -y

sudo apt-get install build-essential ubuntu-desktop -y

3) Next, disable the Nouveau kernel driver, which does not work with the NVIDIA driver. (Only use the NVIDIA driver on NV VMs.) Create a file in /etc/modprobe.dnamed nouveau.conf with the following contents:

blacklist nouveau

blacklist lbm-nouveau

4) Exit PuTTY and reboot your VM via the Azure portal.

 

5) After your VM reboots, log back in to PuTTY and run these commands: (Click "Yes" or "I agree" to the default prompts.)

wget -O NVIDIA-Linux-x86_64-367.106-grid.run https://go.microsoft.com/fwlink/?linkid=849941  

chmod +x NVIDIA-Linux-x86_64-367.106-grid.run

sudo ./NVIDIA-Linux-x86_64-367.106-grid.run

 

6) After that install finishes, copy /etc/nvidia/gridd.conf.template to a new file gridd.conf at location /etc/nvidia/

sudo cp /etc/nvidia/gridd.conf.template /etc/nvidia/gridd.conf

 

7) Add this line to this file: /etc/nvidia/gridd.conf

IgnoreSP=TRUE

8) Exit PuTTY and reboot your VM via the Azure portal.

 

9) After your VM reboots, verify the driver installs by running this command:

nvidia-smi

10) Next, we'll install the CUDA drivers. Download and install the CUDA drivers with these commands:

CUDA_REPO_PKG=cuda-repo-ubuntu1604_8.0.61-1_amd64.deb

wget -O /tmp/${CUDA_REPO_PKG} http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/${CUDA_REPO_PKG} 

sudo dpkg -i /tmp/${CUDA_REPO_PKG}

rm -f /tmp/${CUDA_REPO_PKG}

sudo apt-get update

sudo apt-get install cuda-drivers

11) Give your VM one last reboot and verifiy installation with the nvidia-smi command again:

 

12) Next, download and unzip the Ethminer release with these commands:

wget https://github.com/ethereum-mining/ethminer/releases/download/v0.12.0.dev1/ethminer-0.12.0.dev1-Linux.tar.gz

tar -xvzf ethminer-0.12.0.dev1-Linux.tar.gz

 

13) That creates an ethminer folder that you'll have to change directory into with a a cd command. After you find the ethminer executable, you can kick it off like this:

screen -S ethminer ethminer --farm-recheck 2000 -G -S us1.ethermine.org:4444 -FS eu1.ethermine.org:4444 -O 0xB82adE3cEEE35afcF3415afE983d3E14218A41c3.RigBlogPost

The above command will kick off ethminer using a mining pool at Ethermine.org, which I personally use. Ethminer console output looks like this:

Enjoy Ethereum Mining for fun and profit!

 

 

 

Setup Ethereum and Geth on Ubuntu in Azure - Part 1

Setup Ethereum and Geth on Ubuntu in Azure - Part 1

In Part 1, we'll create an Ubuntu instance in Azure and setup Geth locally. The reason to read this article is that Azure will setup your default home location disk to not be large enough for Geth downloads. Even with a hot NC6 instance (M60 video card, 56 GB ram, and 6 cores), you're default home disk /dev/sda1 is only about 25GB.

 

Most Azure Ubuntu instances will have the larger disk set to /dev/sdb1 so we'll move our home folder there before we install and run Geth (for Ethereum).

 

Let's move our home to the larger disk on /dev/sdb1:

Temporarily mount the new partition:

sudo mkdir /mnt/tmp
sudo mount /dev/sdb1 /mnt/tmp

Copy HOME to the new location:

sudo rsync -avx /home/ /mnt/tmp

Mount the new partition as Home:

sudo mount /dev/sdb1 /home

Unmount the new home and delete the old home:

sudo umount /home
rm -rf /home/*

Make HOME permanent

Find the UUID of the new partition for the fstab entry seen from:

sudo blkid

Copy/paste the correct UUID to edit your fstab via

sudo nano /etc/fstab

Append the following line at the end:

UUID=<copied number from above>    /home    ext4    defaults   0  2

Save and exit nano.

Reboot:

sudo reboot

Wait a couple minutes for the reboot. You'll need to reconnect via PuTTY.

 

Install Geth with these commands:

sudo apt-get install software-properties-common
sudo add-apt-repository -y ppa:ethereum/ethereum
sudo add-apt-repository -y ppa:ethereum/ethereum-dev
sudo apt-get update
sudo apt-get install ethereum

After the geth install completes, kick off geth like this:

geth --fast --cache=1024

Geth should start running and loading the database from peers. This takes between 2 and 5 hours for Geth to sync with the rest of the Ethereum network.

 

As geth runs, you can monitor the disk space usage. Be sure that your data disk is growing and not the original home folder with a small 25GB capacity.

df -h

If your PuTTY connection closes, geth will stop running. Use my article about Screen to let your geth session persist even when PuTTY disconnects. Find that here:

Use Screen to persist PuTTY session commands

 

In future Ethereum article parts, we'll setup the Ethereum Miners and NVidia CUDA for GPU mining.

Use Screen to persist Putty session commands

I spun up an Azure Ubuntu instance and connected with PuTTY from my home Windows box. I want to kick off some commands that run even if my remote PuTTY terminal closes. My specific example uses Geth for Ethereum (an alternative cryptocurrency to Bitcoin).

 

We'll use the Screen command to create a session and allow this:

screen -S sessionname

 

Then, we kick off Geth like this:

geth --fast --cache=1024

 

I can close my PuTTY terminal now, and Geth will continue to run. When I want to open up PuTTY again and resume my session, I can login via a new PuTTY terminal and restore my Screen session like this:

screen -d -R sessionname

 

That will disconnect the previous PuTTY session (if I had left it open), and resume that session on my new PuTTY terminal.

 

Using Function Name, Line Number, and Source File for C# Logging

Using Function Name, Line Number, and Source File for C# Logging

We want a custom logger for C# that automatically helps with function name, line number, and source file name. We can use the following attributes on our function arguments:

  • CallerMemberName
  • CallerFilePath
  • CallerLineNumber

Here's a code sample on how to put it together:

public void LogMessage(string message,
						 [CallerMemberName] string callingFunction = "",
						 [CallerFilePath] string sourceFilePath = "",
						 [CallerLineNumber] int sourceLineNumber = 0)
{
	Debug.WriteLine("message: " + message);
	Debug.WriteLine("function name: " + callingFunction);
	Debug.WriteLine("source file path: " + sourceFilePath);
	Debug.WriteLine("source line number: " + sourceLineNumber);
}

Finally, we can make a call to our LogMessage function like this and the calling function, source file path, and source line number will automatically get populated with the correct values:

LogMessage("INFO: Log message details go here.");

 

Posting with Cookie Containers in C#

Posting with Cookie Containers in C#:

We want to use C# and make a web call Rest POST call to some web API, and we need to add cookie values to our request.

 

We'll extend the basic WebClient C# class like this:

 

    public class MyWebClient : WebClient
    {
        public MyWebClient(CookieContainer container)
        {
            this.container = container;
        }
        public MyWebClient()
        {
        }

        public int Timeout { get; set; }

        public CookieContainer CookieContainer
        {
            get { return container; }
            set { container = value; }
        }

        private CookieContainer container = new CookieContainer();

        protected override WebRequest GetWebRequest(Uri address)
        {
            WebRequest r = base.GetWebRequest(address);
            r.Timeout = Timeout;
            var request = r as HttpWebRequest;
            request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
            if (request != null)
            {
                request.CookieContainer = container;
            }
            return r;
        }

        protected override WebResponse GetWebResponse(WebRequest request, IAsyncResult result)
        {
            WebResponse response = base.GetWebResponse(request, result);
            ReadCookies(response);
            return response;
        }

        protected override WebResponse GetWebResponse(WebRequest request)
        {
            WebResponse response = base.GetWebResponse(request);
            ReadCookies(response);
            return response;
        }

        private void ReadCookies(WebResponse r)
        {
            var response = r as HttpWebResponse;
            if (response != null)
            {
                CookieCollection cookies = response.Cookies;
                container.Add(cookies);
            }
        }
    }

 

Then, we can use the new custom WebClient to assign our cookie name and value pairs and make our POST calls like this:

using (MyWebClient client = new MyWebClient())
{
	client.Timeout = 900000;
	client.Encoding = UTF8Encoding.UTF8;
	client.Headers.Add("Accept-Encoding", "gzip, deflate, br");
	client.Headers.Add("Accept-Language", "en-US,en;q=0.8");
	client.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36");
	client.Headers.Add("Content-Type", "application/json");

	client.CookieContainer.Add(new Cookie(myAuthCookieName, myAuthCookieValue, "/", postDomain));

	string urlResult = client.UploadString(postApiUrl, postData);
	Console.WriteLine(urlResult);
}

It's that easy! Now, you can make POST calls to web APIs with WebClient and include cookie name/value pairs along with your request.