Set up a private Ethereum node on MacOS

Set up a private Ethereum node on MacOS


In previous lectures, we installed
Truffle and Ganache, that can easily
emulate an Ethereum node for development
purposes. While these fake Ethereum nodes
are an easy way to test your application,
you have to keep in mind that they are
just emulators: they provide the same
interface as an Ethereum node
deployed in the public Ethereum network,
but internally they work completely
differently. If you want to ensure that
your contract will run properly on the
main network, it is highly recommended to
test it with a full Ethereum node. In this
lecture, we’ll install a private Ethereum instance with Geth on MacOS. If
you are on Linux or Windows, there are
dedicated videos for those environments.
Let’s start by creating the folder that
will host the database and the accounts
for the private node. So we open a
Terminal, and to create those directories
we are going to use “mkdir -p” to
create all the parent directories, “~”
is for the user directory, for the home
directory, and then we put that in “ChainSkills/private”.
And then we can go
there. So with these commands, the
“~” character represents the user home
directory. So here my username is
“sarbogast” and if I do a “pwd” to print the
working directory, I am in the private
directory. Now before creating our
private instance we have to define the
genesis block, that is the block that
will define the initial behavior of our
blockchain instance. Creating a Genesis
block file from scratch can be painful,
and reusing an existing one can be a
problem if its version is older than
your current installation of Geth. To
ensure that your genesis block file is
compliant with your version of Geth, we’ll
create it using a tool called Puppeth. Puppeth
is a command-line tool that’s installed
with Geth and that provides features to
create
and monitor your own Ethereum network. So now
let’s use that tool to create the
genesis block file. So of course you have
to do that from the private directory,
and then you can run “puppeth”. So first we
have to specify a network name. For our
private network, we’ll call it ChainSkills.
Now we can use one of those commands and
here we’ll use command number 2 to
configure a new genesis. Our private Ethereum node will use a proof-of-work
consensus algorithm, so let’s choose
option 1 for ethash. And we do not want to
define an account to be pre-funded, so we
just type [Enter] on this. Then we have to
define a network identifier for our private
node. This identifier must be different
from those that are used for public
networks. So number 1 is for the main
net. Number 2 is for the Morden test
network, that is now obsolete.
number 3 is for Ropsten test
network. Number 4 is for Rinkeby, and
we will use that later. And number 42 is
for the Kovan test network. In our course
we’ll use the network identifier 4224.
So here we have the option to add
some custom text to the genesis block, we
won’t do that. So we just type [Enter]. So
now we created this genesis block, we
just need to export it. And to do that we
need to choose option 2 in this menu, to
manage existing genesis. And here option
2 again to export the genesis
configuration. By default, it will save it
in a “ChainSkills.json” file, and
we’ll accept the default. And that’s it. As
you can see,
we have this line that says “exported
existing genesis block”. Now you can press
[Ctrl+C] on your keyboard to exit this
tool. And if you list the content of that
directory, you will see the ChainSkills.json
file. And we can open it in Atom
now that we installed it. So here are the
most interesting items in this genesis
file. “config” is a section that is used to
define parameters for the chain, the
network ID used for this instance is
specified in “chainId”. And config also
contains the block numbers from which to
apply various hard-fork rules that have
been added to the Ethereum chain over
time. For example in our case the
Byzantium rules will be applied starting
from block 4 in our private network.
Then “timestamp” is used by the Ethereum
Virtual Machine to adjust the level of
difficulty that must be applied. If the
timestamp difference between two
consecutive blocks is too small, the
difficulty will be increased. If the
timestamp difference between two
consecutive blocks is too high, the
difficulty will be reduced. The timestamp
is also used to ensure that blocks are
in the right order. Then “extraData” can
store any information that we want to
label our blockchain instance. Here we
didn’t set anything. “gasLimit” defines
the maximum value of total gas
transactions in a block can spend. This
value is here to limit the processing
requirements of each block, and in effect
limits the size of blocks. In a private
node you can set this value to the
maximum, while keeping in mind that you
have to measure the gas spent by your
transactions. Difficulty defines the
difficulty level that will be used to
validate a block, which directly
correlates with the average number of times a
miner will have to run the hashing
function to solve a consensus puzzle. On
a private blockchain instance, it is
recommended to set a lower value for
this field to reduce computation time.
Then “coinbase” is the address that will
receive all the mining rewards collected
after successful mining of this genesis
block, and this value is superseded by
the address of the miner when a new
block is created. “alloc” is a section
where we can pre-allocate funds to some
wallet addresses, and here we didn’t do
that. And at the very end of the file,
after the “alloc” section, we can also find
“number”, which is the block number, and in
that case since it’s the genesis block, the
block number is 0. “gasUsed” is the
total sum of all the gas used in this
block. And “parentHash” is the hash of the
parent block. And since we are in a
genesis block, it doesn’t have any
parent,
so this hash is 0. Now that we have
our genesis file, let’s use it to
initialize our private node. Let’s quit
Atom. Now the private node will be
created using the Geth command,
with the following parameters. We need to
specify the data directory, which is used
to set the target directory where the
private node data will be saved. Without
this parameter, the node would use the
folder of the main public node, which is
not what we want. And we’ll also use the
“init” command to specify the genesis
block to initialize the private node.
Again, in this command the “~” character
stands for you user home directory. So
let’s clean up our Terminal, and run the
following command. Again, from within the
private directory, “geth –datadir”
and this will be the “ChainSkills/private” directory within my user
directory, and then “init”, and we can
specify “ChainList.json” since we are
in the directory where this file is
saved. So again “geth –datadir” with the path to store the chain
data, and then “init” and the file. So we
run that command, and that’s it. So at
this stage, you should see two
directories inside the private directory.
Before, we just had the ChainSkills.json file, and now we also have the “geth”
and the “keystore” directory. Geth will
store the blockchain instance data. And
“keystore” will store all our accounts. So
if we just list the contents of “keystore”,
we don’t have anything at this stage. We
only initialized our private instance
but we also need to prepare it for
mining and for usage. The first thing to
do for that is to create a couple of
accounts. In our case, we’ll create 3
accounts that we will use later in this
course. The first one will receive mining
rewards, and the other two accounts will
act as the buyer and/or the seller of
the articles in the ChainList app that
we will build later.
To create those accounts, we will use the “geth”
command, and specify the path of our
private node, so the data directory, and
also the “account new” command. For testing
purposes, we use the same password for
all accounts, that will be “pass1234”.
If you choose another password, please do
not forget it otherwise you will not be
able to unlock the accounts, and your fake
Ether will be lost. So again make sure
that you are inside the private
directory, and here we can do “geth –“
“datadir”. “.” to say the current directory,
and “account new”. Now we have to specify
the password twice, so again we’ll use “pass1234”. And we specify it again. And
that’s it: you just created a first
account. Of course, the address on your
environment will be different. We’ll run
that command a second time, get a
second account. And we’ll do it a third time,
and now we have our 3 accounts. They
are created and they have a unique
address. Of course, these addresses will
be different for you, as they are
randomly generated. And the private keys
of these accounts are located under the
“keystore” directory. So now if I list the
contents of the “keystore” directory again,
I find 3 files that each correspond
to the private key of one of those accounts. And another thing that’s
important is that each account can be
identified by its position in the array
of accounts registered on the node. To
list these accounts in order, you can use
the following Geth command with the
“account list” parameter. So “geth –datadir . account list”
and here we get “Account #0”, “Account #1”,
“Account #2”, with the corresponding
wallet address…
and where the
corresponding file is stored. So far, we
initialized our private node, and we
created 3 accounts for it. But it
didn’t start yet. In order to start our
private node and mine blocks, we will
create scripts that will define all the
required parameters to let us interact
with our private node on a MacOS
environment. We will create a “startnpde.sh”
script that I’m going to show you in a
moment. Just remember that this script is
attached as a resource to this lecture,
so you can download it and use it
directly. I’m just going to go over it
now, to show you what the different
parameters are, and what they are used
for. It’s very important by the way that
all this command line is just
on one line in your file, that there is
no carriage return or line feed, because
otherwise some parameters will be
ignored. So let’s create our “startnode”
file, “startnode.sh”. And of course it
starts with the “geth” command. And we
need first to specify the network ID,
which in our case is 4224. We need to
activate mining, to make sure that this
node starts mining right away.
But we can specify how many threads we
want the miner to use, and in our case
we’ll use “–minerthreads 1” to say that
it should only use 1 thread. If you
have a more powerful machine and you
want to mine faster, you can increase
that number. “–datadir” is used to
specify the directory in which we will
save the data for the chain. Again “~”
is for the user home directory. Then we
want this private node to be isolated,
not to look for peers and connect to
other nodes, so we will deactivate the
discovery
protocol. And to do that “–nodiscover”.
Then we want to enable the RPC endpoint,
because we want to connect to our node
over RPC. That will be useful for Metamask for example. So “–rpc”. We can also
specify the RPC ports: “–rpcport”, and the default one is 8545,
and that’s the one we’ll still use for the
private node. Since we’re here, and even
though the discovery protocol is
deactivated, we can also specify the
discovery port that will be “–port”, and
the default is “30303”.
We want to be able to connect to
our node from any domain so “–rpccorsdomain”
with a star. “–nat” has to do
with the network configuration, so you
just need to specify “any” in double
quotes for that. And then we need to
enable a certain number of modules for
the RPC endpoint, and this is done with
“–rpcapi”, and then we can list all the
modules that we want enabled. So of
course we want “eth”, “web3”. So you separate
all the modules with a comma. “personal”
will be useful as well, and “net”. Also,
since we are going to use the first
accounts as a mining reward recipient,
we’ll unlock it. So “–unlock”, followed by
the index of the account to unlock, which
is 0 in our case. And we want to
specify the password for this account, so
that the geth command is able to unlock
it. So “–password”, and then we specify
the path of the file that will contain
the password: “~/ChainSkills/private/password.sec”,
and we will create this file in
a minute. And the last parameter is
probably one of the most important, and one of those that has the biggest
consequences, is the “–ipcpath’. So when you
start a node, it will create an IPC
endpoint in the form of a file. It’s like
a lock file if you want. And then other
tools like the Geth console, or certain
Dapp browsers and so on will want to
connect to the running node and will
look for a specific IPC file in a
default location. So if we just start the
node without specifying “ipcpath”, it will
create the IPC file in the same
directory, so in our case in the private
directory. What we want is to force Geth
to create this IPC endpoint in the
default location where other tools are
going to look for it. And on MacOS,
because it varies depending on the
operating system, but on MacOS, it will
be in your user directory
under “~/Library/ethereum/geth.ipc”
And this is very
important, because then we will use Geth
to attach a console to our running node
and “geth attach” will by default look for
this file in this location. So if it
doesn’t start in the right location,
we’ll have an issue. One thing that is
very important is that you make sure
that this entire command sits on one
line, and doesn’t contain any line feed
or carriage returns. Otherwise everything
beyond the first line will be ignored,
and you will have issues with your
configuration. So here it’s on one line,
and we can save it. Before we can run the
script, we have to create the “password.sec”
file that will store the
password for the first account. So from
Atom, I will
create a new file in the private
directory by right-clicking “private”, “New
file”, and then I can call it “password.sec”.
And here I can just type “pass1234”, which is the password I
chose for all three accounts that I
created on my private node. And I save.
Again, if you’re not sure about the
content of “startnode.sh”, remember
that this file is attached to this
lesson as a resource, so you can just
download the “startnode” script and run
it as it is. The last thing we need to do
before we can execute it is to make it
executable. And to do that I’m gonna use
“chmod +x startnode.sh”. If you forget this step, you will get a
“permission denied” when you try to run
the script. And now we can start our
private node. So let’s go full screen.
First, I’m gonna quit Atom, show the
bigger window, clear it a little bit,
and then I can run “./startnode.sh”.
I will quit it right away just to show
you the beginning of the logs, because
there are a few very important pieces of
information in there. So we start a
peer-to-peer node and here we can see
the version for Geth that is used here.
We can see that the chain configuration
uses our 4224 chain ID, as well as all
the values that were configured in the
genesis block. Another very important
piece of information is this line: so we
can see that the IPC endpoint was opened
in the “~/Library/Ethereum/” directory
under my user home directory. If you have
something different here, I mean if it’s
not in “~/Library/Ethereum” under
your user home directory, you will have
an issue later, so you need to fix that
first. If you see for example that geth.ipc
is created in your private
directory, you have a problem. That means
there is something wrong with your
“startnode” script. So make sure this is correct.
We can see that the HTTP endpoint was
opened on port 8545 as well. And we see
that the first account was unlocked
successfully. And of course there is one
thing I forgot to show you is that the
database for the node is stored under
our data directory as instructed.
So here all the data files will be saved
under “geth/chaindata”
inside our private directory. So after
all this configuration output,
you can see a series of lines that start
with “Generating DAG in progress”.
Now the DAG, or Directed Acyclic Graph, is
a data structure that is needed by the
Ethash mining algorithm to do its work. It is
generated once every 30,000 blocks
and stored in a file under “.ethash” in
your user directory, which you can see by
the way here: “Disk storage enabled for ethash DAGs”, under your user directory, in
the “.ethash”. Now we’ll leave it
running because generating this DAG the
first time is going to take a little bit
of time, and you can follow the process
by looking at the percentage here. So
here I am on a rather powerful laptop, so
it goes pretty fast, but it can take a
few minutes. So it will do it a first
time for epoch 0. So an epoch is a
period containing 30,000 blocks. So it
will do it for epoch 0, then when it’s
done it will start doing it for epoch
1, and when it’s done for epoch 1, your node will start mining.
So here you can see that it
reached 100% for epoch 0,
and it started doing it for epoch 1,
and at the same time it started mining.
So every time you see a line with
a tool like that, with a hammer or
something that looks like a hammer, it’s
actually because it’s mined a block. Now
you’re better off waiting for the DAG to
be completely generated before you can
use the node. So every time you can see a
line with a pickaxe, I think that’s the
name of the tool, it means that it’s
mining a block. And we’re almost done
with epoch 1, and that’s it. Now the node
is ready and it will start just mining
blocks.
Now if Geth seems to be stuck for several
hours on one of these steps, it might
mean that you forgot to initialize
your node with the genesis file, or you
forgot to create some accounts, or there
is a mistake in your “startnode.sh”,
or your chain data is corrupted for some
reason. So your best option is just to
delete the private directory altogether,
and to do the whole procedure again.
Don’t forget that you can download the
“startnode.sh” script from the
resources attached to this lecture to be
sure that there is no mistake in it, and
if after all that your private node
doesn’t start mining as it should, feel
free to copy/paste as much of your
“startnode.sh” console output into a
question in the Q&A section of this
course, along with the content of your
“startnode.sh” script, and we will
help you troubleshoot what’s going on. At
this stage, there is just one thing that
you might be concerned about: why is
this node mining when it doesn’t contain
any transaction. It’s isolated, I’m not
sending any transaction to it, and yet
it’s creating blocks. You have to
remember that the sole purpose of mining
is not just to mine transactions,
it is also to generate new Ether. So with
a normal proof-of-work
consensus algorithm like the one that
we’ve enabled on this private network, we
are actually generating new blocks at
regular intervals of time, whether there
are pending transactions or not. And each
of these blocks creates a
new quantity of Ether that is assigned
to the miner. And in our case that’s the
coinbase account, the first account that
we created. So don’t be worried if
you see blocks generated even though
there are no transactions: this happens
with every single instance that uses a
proof-of-work algorithm, even on the main
net. So on the public network, we have
blocks that don’t contain any
transaction. So now our node is
mining, so we can try and connect to it
from outside. We can open a new
Terminal. On MacOS, we can just open a
new tab, and from there we can run the
“geth attach” command. And the “geth attach” command
will look for the IPC endpoint in the
default location, and that is again under
“~/Library/ethereum” in my user
directory. And here it found it, so it was
able to connect to my running node, which
I can confirm because it’s the right
version of Geth. Of course, the address
of the coinbase account is the one I
created earlier in this lesson, the data
directory is OK, that’s “~/ChainSkills/private”. So always when you start
“geth attach”, check that the data dir and
coinbase match the ones that we defined
before, otherwise it means that Geth
console attached to another running node,
which by the way you should try
to avoid, so if you already have Mist
or Ethereum Wallet, or another instance
of Geth running on the main net for
example, you cannot start your private
node at the same time because they use
the same RPC port, which is 8545. So
always remember to keep your running
nodes clean. We’ll see later that with
Ganache we don’t have this problem
because it’s running on a different port.
But your private node will be running on
the same port as the main net if you
started Geth or Mist on the main net.
If you get an error like “unable to
attach to remote geth
because geth.ipc there is no such file
or directory”, again that means that the
IPC endpoint was opened in the wrong
directory. So you have to check the
console output for the “startnode.sh”
at the beginning, and make sure that
the IPC endpoint was opened in the right
location.
Now what’s interesting is that, within
this JavaScript console, because geth
attach is a JavaScript console as you
can see here, we can type some commands
that are exposed through one of the APIs
defined in the “–rpcapi” input
parameter. So we can see a list of
modules that we have access to at the
beginning of our console output. And
what’s important to note is that we can
also attach to a node using not the IPC
endpoint, but the RPC endpoint, with an “R”,
and if we do that there are some APIs,
some modules that we won’t have access
to. But with the IPC endpoint, we should
have access to everything. Now here are
some of the useful commands that we can
use within this console. For example,
we can type “eth.accounts” and we get
a list of the accounts that were created
and registered on that node. And here we
can see the first one, the coinbase one
that is used for mining rewards. We
can also by the way use the coinbase
command to get just this account. By the
way when I mean by coinbase
here is really the concept of the main
address that is used for mining rewards.
The coinbase.com website that you
might know is actually named after this
concept, not the other way around. Another
command that we can use is “getBalance()”
so “eth.getBalance()” to get the
balance of an account, and here for
example we can get the balance of
accounts[1] and this one has 0. But
if I do the same thing for accounts[0],
the one that is receiving mining rewards,
already we can see that we have some
Ether there.
Actually, getBalance() returns a number
of wei. Wei is the minimum unit of value
on Ether, so it’s 10^-18
Ether. And there is a command
in the web3 module,
it’s called “fromWei()”, that we can use to
convert from wei to ether. So “eth.getBalance(eth.coinbase)”
account for example, and we can specify
the unit that we want to convert it to,
“ether”, and that’s it. So we see that the
account already has 81 Ether in it. We can
also use “miner.stop()” to stop the
mining process. Now if I go back to the
tab where I have my node running, the
mining has stopped. And if I go back, I
can also restart the miner with “miner.start()”. And I can even specify the
number of threads that I want to use to
mine. So here
the mining process started again…
“starting mining operations”, and it
started mining on 2 threads. Now of
course, if I just use the command without
specifying anything, it will use the
default as configured. Another command I
can use is “net.version” and this will
give me the network identifier, 4224. And
last but not least, I can use the
personal module to manage my accounts,
and more specifically to unlock them. So
let’s say I want to unlock account #1. I
can do it with “personal.unlockAccount(eth.accounts[1])”. Then I need
to specify the password,
and then the timeframe during which I
want to unlock this account. So for
example, if I want to unlock it for 5
minutes, or 300 seconds, I can indicate
300. Now if you wonder what unlocking an
account means, it means that, well
basically, your private key for your
account is always protected with a
password. When you type this password and
you unlock this account, you basically
make the private key available for
signing transactions. While an account
is not unlocked, you cannot use it to
send transactions and you will get an
exception saying “authentication
needed: you need to type your password to
unlock the account”. And if you just type
“personal.unlockAccount()”, and specify
just the name of the account, so for
example if we want to unlock account
number 2, then it will ask for the pass
phrase, and I think the default time
frame is 10 minutes. Oops! So this is
because I made a mistake in the
passphrase, so I need to do it again. “personal.unlockAccount(eth.accounts[2])”.
That’s this. And when you see true, it
means the account is unlocked. Now you
can find a comprehensive list of
commands on this link that I’m
displaying right now. And again this link
will also be in the resources of this
lecture. Now let’s use this geth console
to send some Ether between accounts. So
we’ll transfer some Ether from the
coinbase account, the one that is
currently mining, to the other two
accounts. So again “eth.accounts”
lists all the accounts
and “eth.coinbase” corresponds to the
account that’s receiving mining rewards.
And we also have the balance of the coinbase
account that is now 273 Ether. So to
send cryptocurrency between accounts, we
use the “eth.sendTransaction()” function.
Here we will transfer 10 Ethers from
the coinbase account to the two other
accounts. And then we’ll check their
balance. To do that, we can use “eth.sendTransaction()”, and here we specify all
the parameters in a JavaScript object.
“from” will be the sending address, eth.coinbase.
“to” will be the receiving account,
so “eth.accounts[1]” for example. And
then “value” will be the number of wei
that we want to send from one account to
the other. And for that, we can also use
the “web3.toWei()” function. Before, we used
the “web3.fromWei()” to convert from
wei to Ether, and here we need to do the
opposite. So here we want to convert 10
Ether into wei, and we need to end this
call to “toWei()”. When I send this command,
I get this weird string that is actually
the hash of the transaction that was
created to send the amount. And if I go
to the tab that’s running the node, I can
see that a transaction was submitted, and
there is the hash of this transaction: “b695”, which we can find here: “b695”.
I can run this function again to send
value this time to the second account,
and I get another transaction. And by
the way, you can see that this
transaction was already mined into a
block.
Thanks to that, if we run “fromWei()” on “eth.getBalance(eth.accounts[1])”
for example, this time it contains 10
Ether. And I can do the same thing for
account 2, and for account 0. Well
you won’t actually see the difference
because it kept mining in the
background. Again this works because our
node is mining in permanence, so whenever
we send a transaction to it, it’s mining
it into a block pretty much instantly.
But if mining was stopped, then the
balances would not be updated.
At any point in a geth attach console,
you can always quit by typing the “exit”
command. And now we are back in the
Terminal. Now if you want to stop the
private node while it’s running, you can
just type [Ctrl+C] on your keyboard and
you will see “got interrupt, shutting
down” and “HTTP endpoint closed”, “IPC
endpoint closed”, and everything is
stopped. So in this lecture, we explained
how to create a genesis block file with
Puppeth, we initialized a private node
with the genesis block, we created 3
accounts, we started the private node, we
attached a console, we checked the list
of accounts and their balances, we
started and stopped the mining process,
and we sent Ether from one account to
the others. Note that you can find the
genesis file and the “startnode.sh”
script that is attached to this lesson
as a resource. In the next lecture, we’ll
show you how to install Metamask in the
Chrome browser.

3 comments

  1. i have a problem is when i gave the command for ./startnode.sh it checked everything and an error came saying invalid value "¨8545¨" for flag -rpcport: parse error
    can you plz help

  2. now i have this problem can some one help plz
    INFO [07-17|12:18:38.197] HTTP endpoint opened url=http://127.0.0.1:8545 cors=* vhosts=localhost
    Fatal: Account unlock with HTTP access is forbidden!

Add a Comment

Your email address will not be published. Required fields are marked *