Distributed ledger technology, decentralization, and smart contracts explained
Imran Bashir
Purchase Now !
Just with Paypal
Just with Paypal
Book Details
Price
|
4.00 USD |
|---|---|
Pages
| 647 p |
File Size
|
27,573 KB |
File Type
|
PDF format |
ISBN
| 978-1-78883-904-4 |
Copyright
| 2018 Packt Publishing |
Imran Bashir has an M.Sc. in Information Security from Royal Holloway, University of
London, and has a background in software development, solution architecture,
infrastructure management, and IT service management. He is also a member of the
Institute of Electrical and Electronics Engineers (IEEE) and the British Computer Society (BCS).
Imran has sixteen years' of experience in the public and financial sectors. He worked on
large scale IT projects in the public sector before moving to the financial services industry.
Since then, he has worked in various technical roles for different financial companies in
Europe's financial capital, London. He is currently working for an investment bank in
London as Vice President in the Technology department.
I would like to thank the talented team at Packt, including Ben Renow-Clarke, Suzanne
Coutinho, Alex Sorrentino, Gary Schwartz, and Bhagyashree Rai, who provided prompt
guidance and valuable feedback throughout this project. I am also extremely thankful to the
reviewer, Pranav Burnwal, who provided constructive and very useful feedback that helped
me tremendously to improve the material in this book.
I thank my wife and children for putting up with my all-night and weekend-long writing sessions.
Above all, I would like to thank my parents, whose blessings have made everything possible for me.
About the reviewer
Pranav Burnwal has a background in Research and Development, and he has been working
with cutting-edge technologies for the past few years. The technologies he works on range
from blockchain, big data, analytics (log and data), cloud, to message queues, NoSQL, web
servers, and so on. He has worked across various domains ranging from BFSI, HLS, FMCG,
and automobiles to name a few.
Pranav is an active community member in multiple communities. He is the Regional Head
for Blockchain Education Network (BEN), a registered NGO and a worldwide network of
people of blockchain. He has also organized multiple meetups and a start-up weekend in India.
Pranav has also been an active trainer in the blockchain space for an exciting period of three
years now, for an audience ranging from junior developers to senior VPs. This has also
given him insights into how people understand a new and complex technology, which
helped him frame this book in the best interest of the readers.
Preface
This book has one goal, to introduce theoretical and practical aspects of the blockchain
technology. This book contains all material that is necessary to become a blockchain
technical expert. Since the publication of the first edition of this book, a lot has changed and
progressed further with regards to blockchain; therefore, a need to update the book has arisen.
The multitude of benefits envisaged by the implementation of blockchain technology has
sparked profound interest among researchers from academia and industry who are
tirelessly researching this technology. As a result, many consortia, working groups, projects,
and professional bodies have emerged, which are involved in the development and further
advancement of this technology. The second edition of this book will provide in-depth
insights into decentralization, smart contracts, and various blockchain platforms such as
Ethereum, Bitcoin, and Hyperledger Fabric. After reading this book, readers will be able to
develop a deep understanding of inner workings of the blockchain technology and will be
able to develop blockchain applications.
This book covers all topics relevant to the blockchain technology, including cryptography,
cryptocurrencies, Bitcoin, Ethereum, and various other platforms and tools used for
blockchain development. It is recommended that readers have a basic understanding of
computer science and basic programming experience to benefit fully from this book.
However, if that is not the case then still this book can be read easily, as relevant
background material is provided where necessary.
Table of Contents
Preface 1
Chapter 1: Blockchain 101 8
The growth of blockchain technology 8
Distributed systems 12
The history of blockchain and Bitcoin 14
Electronic cash 14
Blockchain 16
Blockchain defined 16
Peer-to-peer 16
Distributed ledger 17
Cryptographically-secure 17
Append-only 17
Updateable via consensus 17
Generic elements of a blockchain 20
How blockchain works 23
How blockchain accumulates blocks 23
Benefits and limitations of blockchain 24
Tiers of blockchain technology 26
Features of a blockchain 27
Types of blockchain 30
Distributed ledgers 31
Distributed Ledger Technology 31
Public blockchains 32
Private blockchains 32
Semiprivate blockchains 32
Sidechains 33
Permissioned ledger 33
Shared ledger 33
Fully private and proprietary blockchains 34
Tokenized blockchains 34
Tokenless blockchains 34
Consensus 35
Consensus mechanism 35
Types of consensus mechanisms 36
Consensus in blockchain 36
CAP theorem and blockchain 39
Summary 41
Chapter 2: Decentralization 42
Decentralization using blockchain 42
Methods of decentralization 44
Disintermediation 45
Contest-driven decentralization 45
Routes to decentralization 47
How to decentralize 48
The decentralization framework example 49
Blockchain and full ecosystem decentralization 49
Storage 50
Communication 51
Computing power and decentralization 52
Smart contracts 53
Decentralized Organizations 54
Decentralized Autonomous Organizations 54
Decentralized Autonomous Corporations 55
Decentralized Autonomous Societies 55
Decentralized Applications (DApps) 56
Requirements of a Decentralized Application 56
Operations of a DApp 56
DApp examples 57
KYC-Chain 57
OpenBazaar 57
Lazooz 57
Platforms for decentralization 57
Ethereum 58
MaidSafe 58
Lisk 58
Summary 59
Chapter 3: Symmetric Cryptography 60
Working with the OpenSSL command line 60
Introduction 61
Mathematics 62
Set 62
Group 62
Field 62
A finite field 63
Order 63
An abelian group 63
Prime fields 63
Ring 63
A cyclic group 63
Modular arithmetic 64
Cryptography 64
Confidentiality 65
Integrity 65
Authentication 65
Entity authentication 66
Data origin authentication 66
Non-repudiation 67
Accountability 67
Cryptographic primitives 68
Symmetric cryptography 69
Stream ciphers 69
Block ciphers 70
Block encryption mode 71
Electronic Code Book 72
Cipher Block Chaining 72
Counter mode 73
Keystream generation mode 73
Message authentication mode 73
Cryptographic hash mode 74
Data Encryption Standard 74
Advanced Encryption Standard 74
How AES works 75
Summary 79
Chapter 4: Public Key Cryptography 80
Asymmetric cryptography 80
Integer factorization 83
Discrete logarithm 83
Elliptic curves 83
Public and private keys 84
RSA 84
Encryption and decryption using RSA 86
Elliptic Curve Cryptography 86
Mathematics behind ECC 86
Point addition 87
Point doubling 90
Discrete logarithm problem in ECC 93
RSA using OpenSSL 95
RSA public and private key pair 95
Private key 95
Public key 96
Exploring the public key 98
Encryption and decryption 98
Encryption 98
Decryption 99
ECC using OpenSSL 99
ECC private and public key pair 99
Private key 100
Private key generation 100
Hash functions 103
Compression of arbitrary messages into fixed-length digest 103
Easy to compute 103
Preimage resistance 103
Second preimage resistance 104
Collision resistance 104
Message Digest 105
Secure Hash Algorithms 105
Design of Secure Hash Algorithms 106
Design of SHA-256 106
Design of SHA-3 (Keccak) 108
OpenSSL example of hash functions 109
Message Authentication Codes 109
MACs using block ciphers 109
Hash-based MACs 110
Merkle trees 111
Patricia trees 111
Distributed Hash Tables 112
Digital signatures 113
RSA digital signature algorithm 113
Sign then encrypt 114
Encrypt then sign 115
Elliptic Curve Digital Signature Algorithm 115
How to generate a digital signature using OpenSSL 117
ECDSA using OpenSSL 118
Homomorphic encryption 120
Signcryption 121
Zero-Knowledge Proofs 121
Blind signatures 122
Encoding schemes 122
Financial markets and trading 123
Trading 123
Exchanges 124
Orders and order properties 124
Order management and routing systems 124
Components of a trade 125
The underlying instrument 125
General attributes 125
Economics 125
Sales 126
Counterparty 126
Trade life cycle 126
Order anticipators 127
Market manipulation 127
Summary 128
Chapter 5: Introducing Bitcoin 129
Bitcoin 132
Bitcoin definition 134
Bitcoin – a bird's-eye view 135
Sending a payment to someone 136
Digital keys and addresses 143
Private keys in Bitcoin 144
Public keys in Bitcoin 145
Addresses in Bitcoin 146
Base58Check encoding 148
Vanity addresses 148
Multisignature addresses 149
Transactions 150
The transaction life cycle 150
Transaction fee 151
Transaction pools 152
The transaction data structure 152
Metadata 153
Inputs 154
Outputs 154
Verification 155
The script language 155
Commonly used opcodes 155
Types of transactions 156
Coinbase transactions 159
Contracts 159
Transaction verification 160
Transaction malleability 161
Blockchain 161
The structure of a block 161
The structure of a block header 162
The genesis block 164
Mining 167
Tasks of the miners 168
Mining rewards 168
Proof of Work (PoW) 169
The mining algorithm 169
The hash rate 172
Mining systems 172
CPU 173
GPU 173
FPGA 173
ASICs 174
Mining pools 175
Summary 178
Chapter 6: Bitcoin Network and Payments 179
The Bitcoin network 179
Wallets 189
Non-deterministic wallets 190
Deterministic wallets 190
Hierarchical Deterministic wallets 190
Brain wallets 191
Paper wallets 191
Hardware wallets 191
Online wallets 192
Mobile wallets 192
Bitcoin payments 193
Innovation in Bitcoin 195
Bitcoin Improvement Proposals (BIPs) 196
Advanced protocols 196
Segregated Witness (SegWit) 197
Bitcoin Cash 198
Bitcoin Unlimited 198
Bitcoin Gold 199
Bitcoin investment and buying and selling bitcoins 200
Summary 202
Chapter 7: Bitcoin Clients and APIs 203
Bitcoin installation 203
Types of Bitcoin Core clients 204
Bitcoind 204
Bitcoin-cli 205
Bitcoin-qt 205
Setting up a Bitcoin node 206
Setting up the source code 207
Setting up bitcoin.conf 207
Starting up a node in testnet 208
Starting up a node in regtest 208
Experimenting with Bitcoin-cli 209
Bitcoin programming and the command-line interface 211
Summary 212
Chapter 8: Alternative Coins 213
Theoretical foundations 216
Alternatives to Proof of Work 217
Proof of Storage 220
Proof of Stake (PoS) 220
Various stake types 220
Proof of coinage 220
Proof of Deposit (PoD) 221
Proof of Burn 221
Proof of Activity (PoA) 221
Nonoutsourceable puzzles 221
Difficulty adjustment and retargeting algorithms 222
Kimoto Gravity Well 223
Dark Gravity Wave 224
DigiShield 224
MIDAS 225
Bitcoin limitations 225
Privacy and anonymity 225
Mixing protocols 226
Third-party mixing protocols 227
Inherent anonymity 228
Extended protocols on top of Bitcoin 228
Colored coins 228
Counterparty 229
Development of altcoins 230
Consensus algorithms 231
Hashing algorithms 231
Difficulty adjustment algorithms 231
Inter-block time 231
Block rewards 232
Reward halving rate 232
Block size and transaction size 232
Interest rate 232
Coinage 232
Total supply of coins 232
Namecoin 233
Trading Namecoins 235
Obtaining Namecoins 235
Generating Namecoin records 238
Litecoin 240
Primecoin 244
Trading Primecoin 245
Mining guide 246
Zcash 248
Trading Zcash 250
Mining guide 250
Address generation 254
GPU mining 255
Downloading and compiling nheqminer 255
Initial Coin Offerings (ICOs) 257
ERC20 tokens 259
Summary 259
Chapter 9: Smart Contracts 261
History 261
Definition 262
Ricardian contracts 265
Smart contract templates 269
Oracles 270
Smart Oracles 273
Deploying smart contracts on a blockchain 273
The DAO 274
Summary 275
Chapter 10: Ethereum 101 276
Introduction 276
The yellow paper 277
Useful mathematical symbols 278
Ethereum blockchain 279
Ethereum – bird's eye view 280
The Ethereum network 284
Mainnet 284
Testnet 285
Private net 285
Components of the Ethereum ecosystem 286
Keys and addresses 287
Accounts 287
Types of accounts 288
Transactions and messages 289
Contract creation transaction 292
Message call transaction 293
Messages 294
Calls 295
Transaction validation and execution 296
The transaction substate 296
State storage in the Ethereum blockchain 296
The world state 297
The account state 297
Transaction receipts 298
Ether cryptocurrency / tokens (ETC and ETH) 300
The Ethereum Virtual Machine (EVM) 301
Execution environment 303
Machine state 304
The iterator function 305
Smart contracts 306
Native contracts 307
Summary 309
Chapter 11: Further Ethereum 310
Programming languages 311
Runtime bytecode 312
Opcodes and their meaning 312
Arithmetic operations 312
Logical operations 313
Cryptographic operations 314
Environmental information 314
Block information 315
Stack, memory, storage, and flow operations 316
Push operations 316
Duplication operations 317
Exchange operations 317
Logging operations 318
System operations 318
Blocks and blockchain 319
The genesis block 321
The block validation mechanism 322
Block finalization 323
Block difficulty 323
Gas 325
Fee schedule 326
Forks in the blockchain 327
Nodes and miners 327
The consensus mechanism 328
Ethash 330
CPU mining 331
GPU mining 332
Benchmarking 333
Mining rigs 334
Mining pools 335
Wallets and client software 336
Geth 336
Eth 336
Pyethapp 336
Parity 336
Light clients 336
Installation 337
Eth installation 337
Mist browser 337
Geth 340
The geth console 341
Funding the account with bitcoin 342
Parity installation 343
Creating accounts using the parity command line 347
APIs, tools, and DApps 347
Applications (DApps and DAOs) developed on Ethereum 347
Tools 348
Supporting protocols 348
Whisper 348
Swarm 349
Scalability, security, and other challenges 350
Trading and investment 350
Summary 351
Chapter 12: Ethereum Development Environment 352
Test networks 353
Setting up a private net 354
Network ID 355
The genesis file 355
Data directory 357
Flags and their meaning 357
Static nodes 358
Starting up the private network 358
Running Mist on private net 364
Deploying contracts using Mist 367
Block explorer for private net / local Ethereum block explorer 372
Summary 375
Chapter 13: Development Tools and Frameworks 376
Languages 378
Compilers 378
Solidity compiler (solc) 378
Installation on Linux 378
Installation on macOS 379
Integrated Development Environments (IDEs) 381
Remix 381
Tools and libraries 384
Node version 7 385
EthereumJS 385
Ganache 386
MetaMask 387
Truffle 390
Installation 391
Contract development and deployment 392
Writing 392
Testing 394
Solidity language 394
Types 395
Value types 395
Boolean 395
Integers 395
Address 396
Literals 397
Integer literals 397
String literals 397
Hexadecimal literals 397
Enums 397
Function types 398
Internal functions 398
External functions 398
Reference types 398
Arrays 398
Structs 399
Data location 399
Mappings 399
Global variables 400
Control structures 400
Events 402
Inheritance 402
Libraries 403
Functions 404
Layout of a Solidity source code file 407
Version pragma 407
Import 407
Comments 408
Summary 408
Chapter 14: Introducing Web3 409
Web3 409
Contract deployment 410
POST requests 417
The HTML and JavaScript frontend 418
Installing web3.js 419
Example 420
Creating a web3 object 422
Checking availability by calling any web3 method 422
Contract functions 423
Development frameworks 426
Truffle 426
Initializing Truffle 426
Interaction with the contract 435
Another example 438
An example project – Proof of Idea 441
Oracles 454
Deployment on decentralized storage using IPFS 456
Installing IPFS 457
Distributed ledgers 460
Summary 460
Chapter 15: Hyperledger 461
Projects under Hyperledger 461
Fabric 462
Sawtooth Lake 462
Iroha 463
Burrow 463
Indy 464
Explorer 464
Cello 464
Composer 465
Quilt 465
Hyperledger as a protocol 465
The reference architecture 466
Requirements and design goals of Hyperledger Fabric 468
The modular approach 468
Privacy and confidentiality 468
Scalability 469
Deterministic transactions 469
Identity 469
Auditability 469
Interoperability 470
Portability 470
Rich data queries 470
Fabric 470
Hyperledger Fabric 471
Membership services 472
Blockchain services 473
Consensus services 473
Distributed ledger 473
The peer to peer protocol 474
Ledger storage 475
Chaincode services 475
Components of the fabric 476
Peers 476
Orderer nodes 476
Clients 477
Channels 477
World state database 477
Transactions 477
Membership Service Provider (MSP) 477
Smart contracts 478
Crypto service provider 478
Applications on blockchain 478
Chaincode implementation 479
The application model 481
Consensus in Hyperledger Fabric 481
The transaction life cycle in Hyperledger Fabric 482
Sawtooth Lake 484
PoET 484
Transaction families 485
Consensus in Sawtooth 487
The development environment – Sawtooth Lake 487
Corda 491
Architecture 492
State objects 492
Transactions 493
Consensus 493
Flows 493
Components 494
Nodes 494
The permissioning service 494
Network map service 495
Notary service 495
Oracle service 495
Transactions 495
Vaults 496
CorDapp 497
The development environment – Corda 497
Summary 498
Chapter 16: Alternative Blockchains 500
Blockchains 500
Kadena 501
Ripple 506
Transactions 509
Payments related 510
Order related 510
Account and security-related 510
Interledger 511
Application layer 511
Transport layer 512
Interledger layer 512
Ledger layer 512
Stellar 513
Rootstock 514
Sidechain 514
Drivechain 514
Quorum 515
Transaction manager 516
Crypto Enclave 516
QuorumChain 516
Network manager 516
Tezos 517
Storj 519
MaidSafe 520
BigchainDB 520
MultiChain 521
Tendermint 521
Tendermint Core 521
Tendermint Socket Protocol (TMSP) 521
Platforms and frameworks 522
Eris 522
Summary 524
Chapter 17: Blockchain – Outside of Currencies 525
Internet of Things 526
Physical object layer 527
Device layer 527
Network layer 528
Management layer 528
Application layer 528
IoT blockchain experiment 532
First node setup 537
Raspberry Pi node setup 538
Installing Node.js 540
Circuit 543
Government 550
Border control 550
Voting 553
Citizen identification (ID cards) 553
Miscellaneous 554
Health 555
Finance 555
Insurance 556
Post-trade settlement 556
Financial crime prevention 557
Media 558
Summary 559
Chapter 18: Scalability and Other Challenges 560
Scalability 561
Network plane 561
Consensus plane 562
Storage plane 562
View plane 562
Block size increase 563
Block interval reduction 563
Invertible Bloom Lookup Tables 564
Sharding 564
State channels 564
Private blockchain 566
Proof of Stake 566
Sidechains 566
Subchains 566
Tree chains (trees) 567
Block propagation 568
Bitcoin-NG 568
Plasma 569
Privacy 569
Indistinguishability Obfuscation 570
Homomorphic encryption 570
Zero-Knowledge Proofs 571
State channels 571
Secure multiparty computation 571
Usage of hardware to provide confidentiality 572
CoinJoin 572
Confidential transactions 573
MimbleWimble 573
Security 574
Smart contract security 574
Formal verification and analysis 576
Oyente tool 578
Summary 580
Chapter 19: Current Landscape and What's Next 582
Emerging trends 582
Application-specific blockchains (ASBCs) 583
Enterprise-grade blockchains 583
Private blockchains 583
Start-ups 584
Strong research interest 584
Standardization 585
Enhancements 586
Real-world implementations 587
Consortia 587
Answers to technical challenges 588
Convergence 588
Education of blockchain technology 588
Employment 589
Cryptoeconomics 589
Research in cryptography 589
New programming languages 590
Hardware research and development 590
Research in formal methods and security 591
Alternatives to blockchains 591
Interoperability efforts 591
Blockchain as a Service 592
Efforts to reduce electricity consumption 592
Other challenges 592
Regulation 592
Dark side 594
Blockchain research 595
Smart contracts 595
Centralization issues 595
Limitations in cryptographic functions 596
Consensus algorithms 596
Scalability 596
Code obfuscation 597
Notable projects 597
Zcash on Ethereum 597
CollCo 597
Cello 598
Qtum 598
Bitcoin-NG 598
Solidus 598
Hawk 599
Town-Crier 599
SETLCoin 599
TEEChan 599
Falcon 600
Bletchley 600
Casper 600
Miscellaneous tools 601
Solidity extension for Microsoft Visual Studio 601
MetaMask 601
Stratis 601
Embark 602
DAPPLE 602
Meteor 602
uPort 603
INFURA 603
Convergence with other industries 603
Future 604
Summary 606
Another Book You May Enjoy 607
Index 609
Bookscreen
Who this book is for
This book is for anyone who wants to understand blockchain in depth. It can also be used as
a reference by developers who are developing applications for blockchain. Also, this book
can be used as a textbook for courses related to blockchain technology and
cryptocurrencies. It can also be used as a learning resource for various examinations and
certifications related to cryptocurrency and blockchain technology.
What this book covers
Chapter 1, Blockchain 101, introduces the basic concepts of distributed computing on which
blockchain technology is based. It also covers history, definitions, features, types, and
benefits of blockchains along with various consensus mechanisms that are at the core of the
blockchain technology.
Chapter 2, Decentralization, covers the concept of decentralization and its relationship with
blockchain technology. Various methods and platforms that can be used to decentralize a
process or a system have also been introduced.
Chapter 3, Symmetric Cryptography, introduces the theoretical foundations of symmetric
cryptography, which is necessary to understand that how various security services such as
confidentiality and integrity are provided.
Chapter 4, Public Key Cryptography, introduces concepts such as public and private keys,
digital signatures and hash functions with practical examples. Finally, an introduction to
financial markets is also included as there are many interesting use cases for blockchain
technology in the financial sector.
Chapter 5, Introducing Bitcoin, covers Bitcoin, the first and largest blockchain. It introduces
technical concepts related to bitcoin cryptocurrency in detail.
Chapter 6, Bitcoin Network and Payments, covers Bitcoin network, relevant protocols and
various Bitcoin wallets. Moreover, advanced protocols, Bitcoin trading and payments is also introduced.
Chapter 7, Bitcoin Clients and APIs, introduces various Bitcoin clients and programming
APIs that can be used to build Bitcoin applications.
Chapter 8, Alternative Coins, introduces alternative cryptocurrencies that were introduced
after the invention of Bitcoin. It also presents examples of different altcoins, their properties,
and how they have been developed and implemented.
Chapter 9, Smart Contracts, provides an in-depth discussion on smart contracts. Topics such
as history, the definition of smart contracts, Ricardian contracts, Oracles, and the theoretical
aspects of smart contracts are presented in this chapter.
Chapter 10, Ethereum 101, introduces the design and architecture of the Ethereum
blockchain in detail. It covers various technical concepts related to the Ethereum blockchain
that explains the underlying principles, features, and components of this platform in depth.
Chapter 11, Further Ethereum, continues the introduction of Ethereum from pervious
chapter and covers topics related to Ethereum Virtual Machine, mining and supporting
protocols for Ethereum.
Chapter 12, Ethereum Development Environment, covers the topics related to setting up
private networks for Ethereum smart contract development and programming.
Chapter 13, Development Tools and Frameworks, provides a detailed practical introduction to
the Solidity programming language and different relevant tools and frameworks that are
used for Ethereum development.
Chapter 14, Introducing Web3, covers development of decentralized applications and smart
contracts using the Ethereum blockchain. A detailed introduction to Web3 API is provided
along with multiple practical examples and a final project.
Chapter 15, Hyperledger, presents a discussion about the Hyperledger project from the
Linux Foundation, which includes different blockchain projects introduced by its members.
Chapter 16, Alternative Blockchains, introduces alternative blockchain solutions and
platforms. It provides technical details and features of alternative blockchains and relevant platforms.
Chapter 17, Blockchain – Outside of Currencies, provides a practical and detailed introduction
to applications of blockchain technology in fields others than cryptocurrencies, including
Internet of Things, government, media, and finance.
Chapter 18, Scalability and Other Challenges, is dedicated to a discussion of the challenges
faced by blockchain technology and how to address them.
Chapter 19, Current Landscape and What's Next, is aimed at providing information about the
current landscape, projects, and research efforts related to blockchain technology. Also,
some predictions based on the current state of blockchain technology have also been made.