Set up a private Ethereum node on Windows

Set up a private Ethereum node on Windows


In previous lectures, we installed
Truffle and Ganache, that can easily
emulate an Ethereum node for development
purposes. While these fake Ethereum
nodes are an easy way to test your
application, you have to keep in mind
that they are just emulators: they
provide the same interface as an Ethereum
node deployed in the public key
Ethereum network, but internally they work
completely differently. If you want to
ensure that your contract will run
properly on the main network, it is
highly recommended to test it with a
full Ethereum node. In this lecture, we will
install a private Ethereum instance
with Geth on Windows. If you are on MacOS or Linux, please jump to one of the
other lectures that describes the
installation process for your operating
system. Now let’s start a new PowerShell
window to create the folder that will
host the database and all the files for
our private node. Again, I should do
everything in PowerShell on Windows: that
will be easier. And the first thing I
need to do is to create the “ChainSkills/private” directory under my user
home directory. And to do that I can just
run “mkdir” – that’s the PowerShell
command to create a directory, “-p” to
create all the parent directories on the
path of the directory I want to create,
and then I can specify
“ChainSkills/private” since I am in my user directory
already. So now if I go to the “ChainSkills”
subdirectory in my user directory,
I will find the “private” directory under
there. Now, before creating our
private instance, we have to define the
genesis block, that is the block that
will define the initial behavior of our
blockchain instance. Creating a genesis
block from scratch can be painful, and
reusing an existing one can be a problem
if its version is older than your
current installation of Geth. To ensure
that your genesis block file is
compliant with your version of Geth, we
will create it using a tool called
Puppeth. Puppeth is a command-line tool
that is installed with Geth,
and that provides features to create and
monitor your own Ethereum network. So here
we will use Puppeth to create the
genesis block file. First, make sure that
you are in the “ChainSkills/private” directory under your user home
directory, and then you run Puppeth like
that. Again this was installed when we
installed Geth. So, the first thing we
have to do is to choose a name for our
network, and in our case we’ll just call
it “ChainSkills”. And then we have a menu
with several options. In our case we use
command #2 to configure a new genesis.
“Which consensus engine to use?”. We have a
choice between Ethash, which is a
proof-of-work,
or Clique which is a proof-of-authority.
We want it to look like the real thing, so
we’ll use the first option, the Ethash.
“Which accounts should be pre-funded?” We’ll
just leave it like that and type [Enter].
Then we have to choose a chain or
network identifier for our network.
Important thing is that this ID must be
different from the IDs that are already
used for public networks. So the ID #1
is reserved for the main net. ID #2 is
reserved for the Morden test network,
which is now obsolete.
Number #3 is for the Ropsten test
network. Number #4 is for Rinkeby,
another public test network that we will
use later. And 42 is for Kovan, another
test network. In our course, we will use
the network ID 4224, so you can type
4224, and [Enter]. “Anything fun to embed in
the genesis block?” No, thanks. And that’s
it. So now we created the genesis, but we
still need to export it. So it created it
in memory, if you want, and we want to
export it into a JSON file.
To do that, we choose option #2, “Manage
existing genesis’, and then option #2
again for exporting genesis
configuration. Here we can choose a name
for the file, we just keep the default
which is “ChainSkills.json”. And as
you can see we have this that says
“Exported existing genesis block”. And now
our work with Puppeth is done, so we can
quit by just typing [Ctrl+C] on our
keyboard, and that’s it. And we go back to
the prompt. By the way if you type “ls” to
list the content of the directory, you
can see that now we have this
“ChainSkills.json” file. And we can open
it with Atom by just running it like
that: “atom ChainSkills.json”. So here
is the content of the “ChainSkills.json”
file that we will use to initialize
the private node. And here are the most
interesting items in this genesis file.
The “config” is a section that is used to
define the parameters for the chain. The
network identifier is defined in the
“chainId” field, and “config” also contains
the block numbers from which to apply
various hard-fork rules that have been added
to Ethereum chain over time. For example,
the Byzantium hard-fork will be
applied from block number 4 in this
private network. Then we have the
“timestamp” field, and this is used by the
Ethereum Virtual Machine to adjust the
level of difficulty that must be applied.
If the timestamp difference between two
blocks is too small, the difficulty will
be increased. If the timestamp difference
between two consecutive blocks is too
high, the difficulty will be reduced. And
the timestamp is also used to ensure
that blocks are in the correct order.
After that we have the “extraData” field,
where we could have set some custom data.
We didn’t do anything here. “gasLimit”
defines the maximum value of total gas
that transactions in a block can spend.
So this value is here to limit the
processing requirements of each block,
and in effect limit the size of blocks.
In a private node, you can set this value
to the maximum, while keeping in mind
that you have to measure the gas spent
by your transactions. The “difficulty”
field defines the difficulty level that
will be used to validate a block, which
directly correlates to the average
number of times a miner will have to run
the hashing function to solve a
consensus puzzle. On a private blockchain
instance, it is recommended to set a
lower value to reduce computation time.
Then “coinbase” is the address of the
account that will get the mining rewards
for this block. So in this case, we don’t
know yet, so it’s 0. But when this
block is mined, it will get the address
of the miner.
Then “alloc” is a section where we can
pre-allocate funds to some wallet
addresses. And this block is pretty big,
but we can reduce it here in Atom. Then
we have “number”, that is the block number.
And since it’s a genesis block, its
block number is zero. “gasUsed” is the sum of
all the gas used by all the transactions
in this block. There is none, so it’s 0.
And finally “parentHash” contains the
hash of the parent block. And again since
we are in the genesis block, it has no
parent, so that’s the only block where
the parent hash is 0. So let’s leave
Atom. So now that we have our genesis
file, let’s use it to initialize our
private node. The private node will be
created using the geth command with the
following parameters. “datadir” will be
used to set the target directory where
the private node data will be saved,
knowing that, without this parameter, the
node would use the folder for the main
public net. And of course that’s not what
we want. And we’ll also use the “init”
command to specify the genesis block to
initialize the private node. So from
within this “private” directory, under
“ChainSkills”, under the user home
directory, we can use the following
command “geth –datadir”, so we’ll save
all the data in the local…
in the current
directory, and “init ChainSkills.json”. So we’re using the “ChainSkills.json”
as a genesis block to
initialize our node. You can run this
command, and we can see that the data for
the node will be stored under
“geth/chaindata” inside the private directory.
So now we have, in addition to our “ChainSkills.json” genesis file, we also
have a “geth” directory, and a “keystore”
directory. “geth” will store all the data
for the chain, and “keystore” will store
all our accounts. And by the way if you
go to “keystore” and you list the content,
there is nothing. Let’s come back up. So
now we will create 3 accounts that
we will use in this course: the first one
will receive all the mining rewards, and
the other two accounts will act as the
buyer and/or the seller of the articles
in the ChainList Dapp that we will build
later. To create those accounts, we will
use the “geth” command and specify the
path of our private node, and the “account
new” command. So here
“geth –datadir .” – that means the
current directory. And we use the “account new”
Geth subcommand to create a new
account. Your new account is locked with
a password, so each account will have a
private key, and this private key will be
protected with a password, so that it can
only be used to sign a transaction with
the password. So here, for all the
accounts, we are going to use “pass1234”
by default. If you change that password,
don’t forget to adapt it later on. “pass1234”.
We type it a second time, and we
get a new account. And that’s the address
of this new account that we just created.
And we can run this command two more
times. You can use the arrows to recall
past commands. So we run this command
again to create a second account. Each
time it’s
a random one. “pass1234”. And
there is our second account. And we do it
a third time, and that’s it: we have our
3 accounts. Of course, these addresses
will be different for you as they are
randomly generated. And now if you go to
“ls keystore” for example… if you type “ls
./keystore”, you will see that
our “keystore” directory now contains
3 files, one for each account that we
just created. To list the accounts, and
get their position in the list, we can
use the “account list” subcommand. So “geth –datadir” always…
“. account list”. And
then we get a list of accounts, with
their respective index, their address and
the file where they are stored. By the
way, by default the first account, so the
one with index 0, that one, is the one
that will receive all the mining rewards
when we start mining. So, so far we
initialized our private node and we
created 3 accounts for it. But it
hasn’t started yet. In order to start our
private node and start mining blocks,
we’ll create a script that will define
all the required parameters to let us
interact with our private node. Note that
this script, that will be called
“startnode.cmd”, is attached as a resource to
this lesson. So you can download it and
use it as it is. I’ll just go over the
content of this file to explain all the
command-line parameters. So here we’ll
create the file with Atom by using “atom”
and then typing “startnode.cmd. That’s a
command script. And it will open Atom
with this file, this empty file, open for
us to edit. Let’s maximize this, close
the “ChainSkills.json” file. We don’t
need that anymore.
And let’s start typing the command. So of
course it’s “geth” to start Geth, and then
“–networkid” to specify the network
identifier, “–mine” will start this node…
will start the miner on this node right
away, and we can also specify the number
of threads on which we want to mine, and
in that case I’m gonna use “–minerthreads 2”
because I know that I have 2 cores
on this machine that I can use for
mining, but you can choose 1 or
however you want to configure it. But
keep it low otherwise it will eat up all
your processing power on your machine.
Then we can specify the “–datadir”
command-line argument to indicate where
we want to store our chain file, and here
we’ll just put it in the current directory,
since we’ll only run this “startnode”
command script from within the private
directory. Then this node will be an
isolated node so we don’t want it to be
connected with other nodes so we’ll
deactivate the discovery protocol, so
“–nodiscover”. What we want though is to
enable the RPC endpoint, because we’ll
use that to connect to our node using
MetaMask later on, so “–rpc”. And we can
also specify which port we want this
node to listen to so “–rpcport”,
within double quotes “8545”. Then we can
also specify the discovery port, even
though we are not using the discovery
protocol, and that by default is
“30303”. Then we want any
domain to be able to connect to our RPC
endpoints, so “–rpccorsdomain”, that
will be a “*”.
There’s this argument that’s used for
networking: “–nat”. The value will be
“any”. And then we also want to activate a
certain number of RPC modules, and those
are modules for commands that we will be
able to use in the RPC endpoint. So here
“–rpcapi”, and then we can list the
modules separated by commas. So
there will be “eth,web3,personal,net”.
Then we want our node to mine, so we
have to unlock the account that will be
used to receive the mining rewards. And in
that case it will be the account number
#0, so “–unlock”, after which we have
to specify the index of the account to
unlock, “0”. “–password” because of course
you want to unlock this account. We also
need to specify a file that will contain
the password. So here we will create this
file under “password.sec”, and we’ll
create this file in a second. So again
the content of this script is very
critical, so we really advise you to
download the script as it is from the
resources of this lecture, and also make
sure that the whole content of this file
fits on one line. If you put it on
several lines, if you add any line feed
or carriage return to this file, then
everything that’s not on the first line
will be ignored, and then you will have
issues connecting to the node and so on.
So make sure that it all fits on one
line. So here we save the file. And of
course before we can run the script, we
have to create the “password.sec”
file that will store the password of the
first account. So from within Atom, I can
right-click
the private directory here, click
“New file”, and then type “password.sec”
as a name for the file. And in there I
just type “pass1234”, which
is the password for the first account.
And I don’t forget to save that file as
well. Now we can run our node. So first
let’s leave Atom, go back to the
PowerShell, clear the screen a little bit.
And then from within that directory, that
“private” directory, where we just created
our “password.sec” and “startnode.cmd”,
we can run “startnod.cmd” with “./startnode.cmd”, and then it will
start the node. I will shut it down right
away with [Ctrl+C], just to show you the
beginning of the file, “confirm”. And
then if we scroll back up to the
beginning, there are a few very important
elements of information that we can find.
First of all, we can see the entire
command that is run when we run the
script, and that is very important to see
that all the command line parameters
that we specified in the file were
taken into account. Then we can see
“Starting peer-to-peer node on instance
with geth-1.7.3”, that’s the version of Geth
that we are using, so that’s ok. We can
also see that the chain data is
correctly stored under the “geth”
directory, under the “private” directory, in
“ChainSkills”, in our user home directory,
that was the intended place. We can also
see the configuration that we specified
in the genesis file, with the chain ID,
with the homestead block, with the EIP
blocks, and so on, the Byzantium block on
number 4, and so on. All the things with the Ethash DAGs, we’ll talk about it later. And
another thing that’s important is that
we can see here: “HTTP endpoint open
on port 8545″, so
that’s good that’s what we wanted. And
another thing that’s important is this
“IPC endpoint opened on” this obscure file
path. This is important because this is
the default endpoint, and “geth attach”
later will look for this IPC file to
connect to our node. But on Windows we
cannot customize it anyway like on other
systems, so that’s OK. Then we have
confirmation that this account is
unlocked and that corresponds to the
coinbase account as we created it before
for this node. And we can see that right
before we shut it down, it was starting
to commit new mining work. Now let’s
start it again, and you will see that
after this
metadata information about the running
of the node, we can see that it’s
starting to commit mining work. Before it
actually starts mining, it needs to
generate a DAG. Now the DAG, or Directed
Acyclic Graph, is a data structure that
is needed by the Ethash mining algorithm to
do its work. It is generated every 30,000
blocks, which is called an epoch. And it is
stored in a file under
“AppData/Ethash”, and that is what we can see at the
beginning of the logs here:
“Disk storage for Ethash DAGs”, you can see it’s
under
“AppData/Ethash”. So generating this DAG will take
some time the first time, and it may even
take a few hours on some machines. So if
your laptop is old, it doesn’t have a lot
of memory and so on, this might take a
while. But the good thing is that you can
follow the progress of this
procedure by looking at the percentage
field in these lines. And basically it
will do it for epoch #0, and once it
reaches 100% for epoch #0,
it will start doing it right away from
for epoch #1, and it will reach 100%
again. But as soon as it can, it
will start mining. So again, this
will take some time the first time, but
then it should be straightforward.
One important note that you might be
concerned about: why will our node mine,
and why did we already start committing
new mining work when we didn’t send any
transactions yet? Well, first of all, it
has to mine the genesis block, that was
the first one, but then you will see that
the node will start mining blocks
on a regular basis, even though it
doesn’t get any transactions yet. This is
normal, because you have to keep in mind
that the mining process… its role is
not just to mine transactions into
blocks, it’s also to create new Ether,
new currency. And this new currency
should be created on a regular schedule.
So every few seconds for example, a new
block should be created, should be mined
to create a new quantity of Ether,
whether there are pending transactions
or not. So for usual implementations of
this proof-of-work algorithm on Ethereum,
even on the main net, you can find blocks
that are actually empty and don’t
contain any transaction. Now if Geth seems
to be stuck for several hours on one of
these steps, on the DAG generation
process in particular, it might mean that
you forgot to initialize your node with
the genesis file, or you forgot to create
some accounts, or there is a mistake in
your “startnode.cmd”, or your chain data is
corrupted for some reason. So your best
option in these cases is just to delete
the “private” directory altogether, and to
do the whole procedure again. Don’t
forget that you can download the “startnode.cmd” script from the resources
attached to this lecture to be sure
there is no mistake in it. And if after
all that, your private node doesn’t start
mining as it should, feel free to
copy/paste as much of your “startnode.cmd” console output, what we are seeing
now, into a question in the Q&A section
of the
course, along with the content of your
“startnode.cmd” script, and we’ll help you
troubleshoot what is going on. So now we
will just leave it running, and let it go
until the end of the DAG generation
process, and we’ll come back when it’s done.
So as you can see, once it’s reached
100% for epoch #0, it will
start mining and calculating the DAG
for epoch #1 as well. And again, an epoch
is a period of 30,000 blocks. So
here, it has already started mining new
blocks, but the DAG generation process
is going to take most of the processing
power of the machine, and we have to wait
for epoch #1 to be calculated before we
can connect to our node from the outside.
So here we reached the end of the DAG
generation process for epoch #1… reached
100%, and now all our node is doing in
the background is just mining new blocks.
And while it was calculating the DAG
for epoch #1, it already mined 20
blocks or so, and now it’s mining
new ones on a regular basis. So now that
our node is mining, we can attach a
JavaScript console to it. So we’ll leave
it running here, and we’ll open a new
PowerShell window by right-clicking on
the PowerShell button in the task bar,
and then clicking on Windows PowerShell
again. This will open a second
PowerShell window and leave the other
one running. So here, now that our node is
running in the background, we can run
“geth attach” and this will open a JavaScript
console that is attached to our running
node. And here we have a few pieces of
information that we can use to confirm
that we are connected to the right node.
So the “datadir”
is in “ChainSkills/private”, so that’s
OK. We have a certain number of modules
that are available over the IPC
endpoint, and the code base seems to
match the one that is used to mine. In
general, when you want to “geth attach” a
console to a running node, make sure that
you don’t have plenty of nodes running
in the background, like a main net node
using Mist or whatever. Here we only have
our private node running, and we are
connecting to it with the “geth attach”
console. Ganache, that we will run later,
is running on a different RPC endpoint
and different RPC port, so that should be
OK to run in parallel. But the private
node, we shouldn’t have any
other nodes running beside the
private node. Now when you run “geth
attach”, if you get a message like “fatal:
unable to attach to remote geth, timed out
waiting for pipe”, and with more details
after that, it means the private node is
not properly started. So again, check the
content of your “starnode.cmd”, or even
better, download it from the resources of
this lecture, and use it as it is. Now
within this JavaScript console, we
can type commands that are exposed
through one of the APIs, and we will
show you a list of those commands now. So
the first command that we can use is
“eth.accounts”, and this will list all
the accounts that have been registered
on this node. And the first one is “eth.coinbase”, and this is the address. We can
also query the balance of each account
by using “eth.getBalance()” and specify
the account for which we want the
balance as a parameter. So for example, if
we want the balance of the coinbase
account, we can type this command, and
here we can see that we already have
some Ether on it. And if we do the same
thing for account #1 for example: “eth.accounts[1]”
here we have 0. Now what’s
important to understand is that this
“getBalance()” function returns the balance in wei, so that is 10^-18 Ether.
That’s the most elemental unit of value.
And if we want to convert it to Ether, we
can use “web3.fromWei()” as a function, and
pass it “eth.getBalance(eth.coinbase)”
and specify the unit in which we want
to convert. So here we see that we have
147 Ether on this coinbase account. Now
you can also use “miner.stop()” to stop
the mining process. And now if we go back
to the other console, we can see that the
mining process stopped. And we can start it
again, and we can even specify the number
of threads that should be used to mine.
So here I’m mining on 1 thread again.
“Updated mining threads=1”,
“Started mining operation” and now it’s
it is gonna mine slightly slower,
because it’s only using 1 thread.
By the way, you can find a comprehensive
list of commands on this link, and that
link will also be in the resources of
this course. I’m just gonna show you a
couple more, so let’s go back to the
console. You can also use “net.version”
to get the network identifier. And you
can use functions from the personal
module to unlock accounts. Now unlocking
an account means basically opening the
private key for access to sign
transactions. By default, private keys are
locked, and they can only be used if you
know the password and you can unlock
them for a
certain duration, so that they can be
used to sign transactions. By the way,
when we start the node, we unlock the
account with index #0 to receive mining
rewards, and be able to write blocks in the
chain. Now let’s say we want to unlock
account with index #1. You can type
“personal.unlockAccount(eth.accounts[1])”, then we need to specify the
password and the duration for which we
want to open it. So for example 300
seconds, that will be 5 minutes. And if we
get “true”, that means the account has been
unlocked. We can also do it more simply
by typing “personal.unlockAccount()”
and this time we’ll unlock account with
index #2, without any other parameter. And
there, it will ask for the pass phrase
for this account, so “pass1234”, and the
default duration I think is 10 minutes.
Now before closing this lecture, we will
transfer some Ether from the coinbase
account, the one that is linked to the
mining process, to the two other accounts.
To transfer 10 Ether from the coinbase
account to the other 2 accounts, we’ll use
the “eth.sendTransaction()” function, so
“eth.sendTransaction”, and that
function takes only one parameter in the
form of a JavaScript object. And this
JavaScript object will have 3 fields.
The first one is the “from”, which is the
address from which to send the funds, so
“eth.coinbase” in our case. Then we
have a “to” field, that will contain the
address of the destination account.
And then we have the “value” field, that
will contain the number of weis that we
want to send from account #0 to account #1.
And to do that, instead of doing some
complex calculation, we can also use the
“web3.toWei()” function. Earlier, we
used the “fromWei()” function. Here we can
use the “toWei()” one, and we want to
transfer 10 Ether from one account to
the other. And when we send that command.
What we get here is the transaction hash,
so a new transaction was created and
sent to the node. And if we go back to
the other window, we can see that the
transaction was submitted, and as a
result it was included in one of these
next blocks. We’ll run this command again
to send 10 Ether to account with index
2. So we can just call back previous
commands with the arrow keys, so [Arrow up]
will call back the last command, and this
time we want to send from coinbase to
accounts with index 2 the same amount.
And another transaction is created and
will be mined, and here is the other
transaction. So now we’re assuming that
both of those transactions have been
mined into blocks, so we should be able
to get updated balances. So again we can
call back previous commands. So for
example “accounts[1]” now has 10 Ether,
and “accounts[2] has 0 Ether as of now.
So that means the second transaction has
not been mined yet. Let’s do it again. There’s something wrong here. Let’s
wait a little more, and there we go. Now
account #2 has 10 Ether. So again,
keep in mind that if the mining process
is stopped when you send transactions to
it, they will not be mined into new
blocks, so they won’t have any effect on
the chain itself.
The last thing we need to do is to exit
this Geth console, and to do that we can
just type “exit” and [Enter].
We can close this PowerShell
window. And then to stop the running node,
you can simply type [Ctrl+C] on your
keyboard, “confirm if you want to
terminate the batch job”, and you can just
type [Enter] or [Ctrl+C] again. So in this
lecture, we explained how to create a
genesis block file with Puppeth, we
initialized a private node with the
genesis block, we created 3 accounts,
we started the private node, we attached
a Geth console, we checked the list of
accounts and their balances, we started
and stopped the mining process, and we
send some Ether from the coinbase
account to the other two accounts. Again,
don’t forget that you can find the
genesis file and the “startnode.cmd”
script attached as resources to this
lesson. In the next lecture we’ll show
you how to install MetaMask in the
Chrome browser.

12 comments

  1. Hopefully this comment helps someone. I was getting an error [Fatal: Unable to attach to remote geth: no known transport for URL scheme "c" ], when I did "geth attach" for the Geth JavaScript console in the new PowerShell window. The following command made me attach my console properly:

    ..ChainSkillsprivate> geth attach ipc:\.pipegeth.ipc

  2. when i typed miner.stop() the result was "null". Then I typed miner.start() and nothing happens.There was no "null" returned. It just froze the PS screen. Whats weird is that I went back to the mining PS screen and just did Cntl-C, then I saw mining resumed normally. It was like in the background, the other PS screen accepted the miner.start() command, but it just didn't return a message. I had to Cntrl-C on the mining PS screen to see it resume !

  3. I'm facing problem . Should the precompile-addresses (0x1 .. 0xff) be pre-funded with 1 wei? (advisable yes) after
    Should the precompile-addresses (0x1 .. 0xff) be pre-funded with 1 wei? (advisable yes)

  4. Hi,
    When I run the file startnode.cmd I get an error :account unlock with Http access is forbidden. Why is that sir and what should I do ?

Add a Comment

Your email address will not be published. Required fields are marked *