A Somewhat Disappointing Description of Bitcoin

The article "What Is the Blockchain?" appeared in the April 2018 issue of IEEE Computing Edge, authored by Massimo Di Pierro.  It also originally appeared in the September/October 2017 issue of IEEE Computing in Science & Engineering.

I was somewhat surprised and disappointed that this article was published in its current form.

The section describing what a Bitcoin fundamentally is and how it is created is not correct.  It is incorrect even when you allow for simplification in order to explain it to a reader without specific domain experience in this area.

The text of this section is as follows:
"A unit of bitcoin is nothing other than a number, but only some numbers are valid bitcoins.  These numbers are solutions solutions of a well-defined equation, and whoever finds a new solution owns it (this process is called mining).  Once a bitcoin is discovered, it can be traded, with transactions stored in a ledger.  Transactions are digitally signed with the credentials of the seller to avoid nonrepudiation.  There is no centralized ledger because users wouldn't trust one and because there are too many transactions to store them all in one place.  Hence bitcoin and other cryptocurrencies provide a distributed ledger in which every computer involved in the transaction of a specific coin (or fraction of a coin) keeps a copy of the history of that coin's transactions.  The blockchain technology makes sure that no party storing this history can tamper with it without being detected."


1) Each Bitcoin not assigned a number.  And they are not created by finding one of these magic numbers.  While this description does align with the term "mining" (which to me has never seemed like an accurate description of what was actually occurring), it is not correct.

    a) "Mining" is actually a race to validate the next set of transactions, gathered in a block.  The kicker here is that a successful "mining" operation, in addition to validating the transactions, also has to calculate a qualifying hash (using the hash function SHA256) of the block.  The block header has fields built in that can be iterated upon in order to vary the hash result.  The current race is won when a "miner" has successfully found a qualifying hash value.  As more computing power is devoted worldwide to mining, the calculation difficulty of what counts as a "qualifying" hash is periodically adjusted (every 2016 blocks) so that rate at which new blocks are turned out remains at approximately once every 10 minutes.  Bitcoins are created when the winner of the current block race assigns himself a number of Bitcoins through an additional transaction (called the coinbase transaction).  This reward was originally 50 per block but is currently 12.5 and is cut in half every 210,000 blocks, which is approximately every four years.  This is the established reward for doing the block validation work and winning the current validation race.

    b) Bitcoins are not individually identified - they are only identified as quantities in transactions.  Therefore, you can't actually trace a bitcoin in the system, only the transactions.  This becomes even more evident when you pay attention to the fact that the smallest unit of currency the system will transact is not a bitcoin, but rather 1/100,000,000th of a bitcoin, which is known as a satoshi.

2) There is in fact a centralized ledger.  Not centralized in that there is only one copy, but centralized in that all transactions in the system can be stored on a single node.  Note that transactions are stored, not the Bitcoins themselves.  A person (wallet id, actually) can create outgoing transactions to spend Bitcoins (or fractions thereof) by referencing incoming transactions.  While not all nodes on the Bitcoin network maintain a full, up-to-date copy of the Bitcoin blockchain, many do.  These are full nodes and they can autonomously and authoritatively verify any transaction without external reference.  The size of the full blockchain can be found by conducting an internet search (ex. "bitcoin full blockchain size").  One site listing the historical size is: https://www.statista.com/statistics/647523/worldwide-bitcoin-blockchain-size/ .  According to this site, "The size of the Bitcoin blockchain has been growing since the creation of the Bitcoin virtual currency in 2009, reaching approximately 173 gigabytes in size by the end of June 2018."


Additionally, not worth mentioning on its own but as long as we are here, there is a typo in the article author's first Python function code example.  The function converts the parameters into a combined "token" object, but then does does not use that object in the hash function computation, instead using one of the function's input parameters.


For those wanting a whole book's worth of how Bitcoin works, I recommend the book "Mastering Bitcoin: Programming the Open Blockchain" by Andreas M. Antonopoulos.

Will the Real Hybrid Cloud Please Stand Up?

A long, long time ago, so long ago that the distance back can be measured in years, not weeks or months, Hybrid Cloud meant a deployment model that was a combination of private and public cloud that were bound together.  Private would often mean on premises.  This was part of the definition, as written by NIST:  http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf

However, the term hybrid has been acquired for use to also mean deployment across heterogeneous environments, such as different public cloud providers.  An episode of Enterprise DevOps Initiatives podcast this past summer titled "Hybrid Cloud is Not a Choice, It’s a Realization" (https://www.cloudtp.com/doppler/hybrid-cloud-not-choice-realization/) has a title that seems somewhat provocative when taken in context of the earlier definition.  Private clouds tend to be mandated by a rather specific requirement, such as data location regulations, and are not necessarily a common necessity.  So to state that it is a realization to eventually come to seems somewhat hyperbolic.  However, in the context of the newer definition, the title is much more mainstream. 

Services offered by major public cloud providers such as Amazon, Google and Microsoft have become complex and varied beyond the basic storage, compute and network they started with.   A quick look here: https://cloud.google.com/products/ or here: https://aws.amazon.com/products/ shows this.  So the argument they are making is that in order to make good use of cloud services, you end up using many of the products.  And inevitably, the differences in available products will cause you to customize for one public cloud provider or another.  This will be true even for service products that are equivalent across cloud providers, as there will be some variation in implementation.  According the discussion between the host, Mike Kavis, and the guest, Rob Hirschfeld, this definition of hybrid would extend to differences in software versions of the same product if the version differences required migration problems for the application using it.

One attempt at mitigation of this is to only use the basic cloud services - such as compute, storage and networking mentioned above.  However, doing so loses many of the advantages of a cloud based implementation.

The guest suggests that hybridization cannot be eliminated, but simply must be managed.  The approach advocated, as can be seen at his company RackN https://rackn.com/ and the associated project Digital Rebar http://rebar.digital/ , is to compartmentalize the variations between platforms so that changes required for portability are isolated.

In another episode from a different podcast, Cloud Weekly Podcast, titled "Vendor Lock-In is Unavoidable" (https://www.cloudtp.com/doppler/vendor-lock-unavoidable/) the discussion echoes some of the same ideas.  Namely, that using the comprehensive set of services available from a cloud provider gives great benefits, but also leads to being tied to that provider.  The solution is, again, plan for it and manage it.

Now, before we shed a tear for the passing of the previous definition of hybrid cloud computing, observe that it is still in use.  Last week there was announcement of a partnership between VMware and AWS: https://blogs.vmware.com/vsphere/2016/10/vmware-aws-announce-strategic-partnership.html . Note this sentence from the announcement: "Currently in Technology Preview, VMware Cloud on AWS, will bring VMware’s enterprise class Software-Defined Data Center software to the AWS cloud, and will enable customers to run any application across vSphere and VMware Cloud Foundation based private, public and hybrid cloud environments."

Additionally, the original definition of hybrid cloud becomes more nuanced as we move forward.  Microsoft spent a significant amount of time discussing hybrid cloud at its recent Ignite Conference.  The graphic at the beginning of this article reporting on Microsoft's attention to the subject - http://www.zdnet.com/article/why-microsoft-is-circling-its-hybrid-cloud-wagons/ - shows that the concept of hybrid cloud is being stretched to include public cloud services combined with on premises servers (ex. Hybrid Database = Azure SQL Database + SQL Server; Hybrid Identity = Azure Active Directory + Active Directory).  This interpretation makes sense, of course, for Microsoft as it plays to their established strengths of existing enterprise software.

So, will the real Hybrid Cloud please stand up?

The Bonus OpenSSH – Or Not?

Previously, I connected into the OpenSSH server that became active by default if you activated Developer Mode in Windows 10 and were running a recent enough version.  This, however, connects into a standard Windows cmd shell which is not what I was hoping for.  So, with a goal of connecting to the WSL Bash command line there are a couple of different routes that immediately come to mind.

One route is to attempt an install of openssh-server in the WSL.  The method for this is pretty straightforward.  Root shell and apt-get install openssh-server.  Install starts out with no surprises, packages pulled down, unpacked, keys created.  Right until it attempted to have Upstart kick things off.  That is when we hit the wall.  The relevant error messages spit out during the installation were:

initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused

runlevel:/var/run/utmp: No such file or directory

invoke-rc.d: policy-rc.d denied execution of restart.

So the services needed to get this executing are not running and are not being started on demand.  This can be further verified by doing a process list, where the only background process running is init, and this init is kicked off with the command /init, not /sbin/init as would be normally expected.  Furthermore, /sbin/init does exist (along with many of the other files that would be expected in /sbin) but it is different than the one running.  Is this because the system calls needed to handle this are not yet implemented?

In a series of videos explaining the WSL (Windows Subsystem for Linux) architecture here:

https://blogs.msdn.microsoft.com/commandline/2016/06/02/learn-more-about-bash-on-ubuntu-on-windows-and-the-windows-subsystem-for-linux/

they talk generally about how Linux processes work with the NT kernel and how support has been implemented.  Which is basically that they have a driver in the kernel that handles Linux system calls.  So, there are two items of note relating to this.  1) In the video on architectural overview, Deepu Thomas mentions that sshd requires a good pseudo terminal implementation, which is not yet done. (Although this particular video was recorded back in April, so you can always hope they have made further progress.)  2) In the video on WSL system calls, Stephen Hufnagel does an estimate of their progress on Linux system call implementations.  He says that out of approximately 300 system calls, they have implemented somewhere in excess of 150.  The rest they are still working on.  So, have they yet implemented the ones required for the particular services we are interested in?  I guess I’ll circle back to that.

Another route to getting a Bash shell via ssh is to go through the existing openssh server.  Since you can run bash.exe from the cmd command line and ssh’ing in drops you into a cmd command line, can you jump to Bash from there?  Attempting to execute bash.exe after connecting with ssh spits out the message:  Error: 0x80070020.  Attempting it again at a different time:  Error: 0x80070005.  Either way, no dice.  A little more investigation leads to a comment in a bug report and a surprise (to me):

Closing this issue for now since, as mentioned above, this SSH server is only a minimal SSH server used for device discovery scenarios and it doesn't support all the high-fidelity SSH features one would expect or rely upon.

If you want to run an OpenSSH Server on Windows, know that the real OpenSSH is currently being ported to support Windows as we type: https://github.com/PowerShell/Win32-OpenSSH

Link bug report: https://github.com/Microsoft/BashOnWindows/issues/777

So, maybe our bonus OpenSSH server is not the bonus previously assumed.  Connecting in via ssh and using the –v flag to get version information (among other things), reveals that the reported version of the default ssh server enabled with the Developer Mode turned on in Windows 10 is “Remote protocol version 2.0, remote software version MS_1.100”.  Huh.  The version reported by an ssh server running on XUbuntu 14.04 gives “Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8”.  So, delving into the above GitHub project - releases, wiki and installation instructions - it would appear that the default ssh server is not that one.  Evidently, not quite the assumed bonus.  And we all know what happens when you assume.

The Bonus OpenSSH

So, continuing to determine how far I can get with the Windows Subsystem for Linux (WSL) -  Previously, it was pretty straightforward to clone in a Git repository that resided on another machine running XUbuntu 14.04.  Unfortunately, cloning out a repository living in WSL did not work as well.  The default protocol I used went over ssh and didn’t connect in.  Given that I hadn’t installed sshd yet, this is not entirely surprising.  But I thought perhaps it was installed by default.

 

Poking around trying to find what network services are available showed what is not.

Doing an ifconfig from the bash shell returned the message:

The command could not be located because '/sbin' is not included in the PATH environment variable.

This is most likely caused by the lack of administrative privileges associated with your user account.

ifconfig: command not found

Switching to root and repeating ifconfig yielded a different message, but still not the data being searched for:

Warning: cannot open /proc/net/dev (No such file or directory). Limited output.

Neither was ping available from the bash command line:

ping: icmp open socket: Permission denied

Apparently, network services in WSL are still limited.

So, while testing an ssh connection in from another machine to see if anything was already listening led to an unexpected discovery.  The was an ssh server receiving connections and these connections were going to the standard Windows command line.  It was using the Windows login usernames and credentials but not the usernames from the WSL.  Doing a little online searching revealed the project behind this.  Here is the initial announcement from Microsoft:

https://blogs.msdn.microsoft.com/powershell/2015/06/03/looking-forward-microsoft-support-for-secure-shell-ssh/

And then a later announcement of public availability of the code:

https://blogs.msdn.microsoft.com/powershell/2015/10/19/openssh-for-windows-update/

Note that while it is on by default, it would seem that it is only installed if you enable Developer Mode (Settings->Update & Security->For developers) and also if your OS is up to Version 1607 (for Windows 10 Home edition).  I have a Windows 10 machine still on version 1511 and enabling Developer Mode did not cause OpenSSH to be running. (Neither did it provide for the option of turning on Windows Subsystem for Linux.)

So I guess we’ll have to see what sort of conflict arises by trying to install sshd within the WSL.

Getting Started with Bash on Ubuntu on Windows

One of the new features in Windows 10 is the Bash shell.  More accurately, “Bash on Ubuntu on Windows”.  I have seen mixed opinions about whether this is a good thing or not.  However, I think it is without a doubt a significant improvement for server side developers.  Previously, I would have only used XUbuntu or OSX for server side development for both the seamlessness of ssh connections and support for various tools (such as command line Git).  Now, doing this on Windows is a reasonable option.

In the not very distant past, I was working on a team doing some Node.js development on Ubuntu.  Part of the group was inexperienced on several of the key tools, including Git.  They were also the part using Windows for this development.  This led to using the Git graphical interface on Windows, which compounded the lack of understanding what was going on under the hood in Git and ultimately led to some problems.  A regular “UNIX style” command line would have gone a long way to ameliorating this situation and bringing them up to speed.

Back in the present, installation of the Ubuntu subsystem was pretty straightforward.  There are various pages found easily enough that explain what to do.  During the installation, it asks you for a UNIX username and password.  I had seen one installation instruction page that said this was root account, but I think they just misused the term.  It is a regular account with sudo privileges, just like a standard Ubuntu installation.

From a version standpoint, ‘lsb_release –a’ reveals it is “Ubuntu 14.04.4 LTS” with ‘uname –a’ giving the kernel release at 3.4.0+ .  That seems pretty reasonable considering 1) when this subsystem first came out for Windows and 2) Ubuntu 14.04.4 was out in February 2016 while 14.04.5 was out in August 2016 according to the release schedule here:  https://wiki.ubuntu.com/TrustyTahr/ReleaseSchedule . 

In gathering information from the Bash window, I noticed that the copy and paste facility is a bit clunky.  Selecting text for copying only works if the window is put into what appears to be “Mark” mode by using the right-click popup menu.  This appears to be the same as how it works in a standard Windows command line window.  The paste mechanism works ok from the popup menu, but ctrl/v does not work, although in a standard Windows command line the ctrl/v key combination does work correctly.

Starting with Git: 

I did an apt-get install git from a root shell which went entirely as expected.  The version that installed was 1.9.1.  Since the version that just installed on a machine running XUbuntu 16.04 LTS was 2.7.4, it is of course not the most current. However, it is the version you would expect to appear on 14.04.4.  It is the same version that apt-get tells me is the newest version on a machine running XUbuntu 14.04.5.  Anyway, cloning a repository onto the Bash subsystem went as expected.

Follow on thoughts:  How well will this work as a server?