December 22, 2016
Lessons Learned After Operating a Real-Life Blockchain Product
We kicked off the engagement by doing a review of the different technology options available. We looked at both open and proprietary frameworks, and finally decided to use the Ethereum blockchain.
At that point, the Ethereum project was still in alpha and largely unknown to the public. However, it had a few great things going for it:
- Open source, with a good number of tools and libraries available
- Support for Smart Contracts
- Good documentation that was well thought out for development
We realized early on that existing wallets would not give us the flexibility and ease of use that we needed. We wanted web access instead of an installable application running a local node, which is the default Ethereum set up. We decided to build a hybrid application: a web app for traders integrated with our own set of Ethereum nodes.
The product was launched in May and has had several major upgrades since then. After one year of development and multiple months of operations, this is our list of learnings and recommendations:
Plan for steep learning curves
Each piece in the technology stack comes with its issues and limitations. You need strong developers that can pick them up quickly, and a plan that gives them the time to do this well. There will be a lot of refactoring over time as your team climbs the learning curve. You also have a low chance of finding engineers with this kind of experience on the market, so you should look for development partners instead.
Usability vs. Security
Blockchain is based on the use of a private key. In a basic Bitcoin wallet, if you lose this private key then you lose your coins and there is no way to recover them. This is an unfeasible solution for any large-scale operation, because users will lose their keys now and then.
There are ways to go around this issue without stepping out of blockchain - for instance, you may have an admin account that can reset user accounts. But these options introduce additional vulnerabilities in terms of security, and may not be very usable. The further you go from the blockchain model, the more conventional security issues you will face.
Infrastructure is a big deal
Running multiple Ethereum nodes in a production setting is very involved. Hard forks require upgrading the nodes on short notice, otherwise they get disconnected from the network. Recently Ethereum went through several DDoS attacks and suffered both transactions processing slowly and nodes falling behind. You may also need a parallel production environment to fully simulate some network conditions that are not present in testing.
Overall, you need strong DevOps resources and an investment in automation to make it work. Many companies are now offering cloud-based Ethereum, which should dramatically improve this area.
Smart contracts are experimental technology
Ethereum is one of the first platforms to support Smart Contracts. Spending time upfront designing your contract structure and events will pay off, because any future change will trigger a lot of dependencies. Instead, think of it as designing your API. As a first-iteration technology, Smart Contracts come with many limitations, which we will explore in a future post. Some of these issues will go away as Ethereum evolves, but in the meantime, you will want to minimize the amount of logic you put into these contracts.