title	excerpt
P-Rep Installation and Configuration	General information about the ICON P-Rep election - https://icon.community/iconsensus/

This document is a guideline about how to install and operate a Public Representative (“P-Rep”) node on the Testnet using a docker. P-Reps are the consensus nodes that produce, verify blocks and participate in network policy decisions on the ICON Network. The purpose of the Testnet is to provide a test environment for the P-Rep candidates. The Candidates can simulate various activities such as block production to check technical stability of the nodes on the Testnet. (The node operation guideline on the ICON Mainnet for the elected P-Reps will be released in the future.)

Intended Audience

We recommend all P-Rep candidates to go through this guideline and participate in the block generation test.

Pre-requisites

We assume that you have previous knowledge and experience in:

IT infrastructure management
Linux or UNIX system administration
Docker container
Linux server and docker service troubleshooting

HW Requirements for Testnet

Below specification is a minimum requirement for the testnet application.

Description	Minimum Specifications	Recommended Specifications
CPU model	Intel(R) Xeon(R) CPU @ 3.00GHz	Intel(R) Xeon(R) CPU @ 3.00GHz
vCPU (core)	16	36
RAM	32G	72G
Disk	200G	500G
Network	1Gbps	1Gbps

SW Requirements

OS requirements

Linux (CentOS 7 or Ubuntu 16.04+)

Package requirements

Docker 18.x or higher

For your reference, ICON node depends on the following packages. The packages are included in the P-Rep docker image that we provide, so you don't need to install them separately.

Python 3.6.5 or higher
RabbitMQ 3.7 or higher

Network Diagram of P-Rep nodes

Above diagram shows how P-Rep nodes are interacting with each other in the test environment.

Endpoint: https://zicon.net.solidwallet.io
- Endpoint is the load balancer that accepts the transaction requests from DApps and relays the requests to an available P-Rep node. In the test environment, ICON foundation is running the endpoint. It is also possible for each P-Rep to setup own endpoint to directly serve DApps (as depicted in PRep-Node4), but that configuration is out of the scope of this document.
Tracker: https://zicon.tracker.solidwallet.io
- A block and transaction explorer attached to the test network.
Firewall
- Port 7100: Used by gRPC for peer to peer communication between nodes. (We recommend any open)
- Port 9000: Used by JSON-RPC API server. (We recommend any open)

Inside a P-Rep Node

A process view of a P-Rep node

There are five processes, iconrpcserver, iconservice, loopchain, loop-queue, and loop-logger.

iconrpcserver
- iconrpcserver handles JSON-RPC message requests
- ICON RPC Server receives request messages from external clients and sends responses. When receiving a message, ICON RPC Server identifies the method that the request wants to invoke and transfers it to an appropriate component, either loopchain or ICON Service.
iconservice
- ICON Service manages the state of ICON network (i.e., states of user accounts and SCOREs) using LevelDB.
Before processing transactions, ICON Service performs the syntax check on the request messages and prevalidates the status of accounts to see if the transactions can be executable.
loopchain
- loopchain is the high-performance Blockchain Consensus & Network engine of ICON.
loop-queue (RabbitMQ)
- RabbitMQ is the most widely deployed open source message broker.
- loopchain uses RabbitMQ as a message queue for inter-process communication.
loop-logger (Fluentd)
- Fluentd is the open source data collector, which lets you unify the data collection and consumption.
- Fluentd is included in the P-Rep node image. You can use Fluentd to systemically collect and aggregate the log data that other processes produce.

Which ports a P-Rep Node is using?

For external communication:

TCP 7100: gRPC port used for peer-to-peer connection between peer nodes.
TCP 9000: JSON-RPC or RESTful API port serving application requests.

For internal communication:

TCP 5672: RabbitMQ port for inter-process communication.

For RabbitMQ management console:

TCP 15672: RabbitMQ Management will listen on port 15672.
- You can use RabbitMQ Management by enabling this port. It must be enabled before it is used.
- You can access the management web console at http://{node-hostname}:15672/

P-Rep Installation using Docker

Please read the SW requirements above. In this chapter, we start by going through the docker installation.

If you already have installed docker and docker compose, you can skip the part below, and directly go to the Running P-Rep Node on Docker Container

Prerequisites - Docker & Docker Compose Installation

If you don't already have docker installed, you can download it here: https://www.docker.com/community-edition. Installation requires sudo privilege.

On Centos 7

Step 1: Install Docker

## Install necessary packages:
$ sudo yum install -y yum-utils device-mapper-persistent-data lvm2

## Configure the docker-ce repo:
$ sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

## Install docker-ce:
$ sudo yum install docker-ce

## Add your user to the docker group with the following command.
$ sudo usermod -aG docker $(whoami)

## Set Docker to start automatically at boot time:
$ sudo systemctl enable docker.service

## Finally, start the Docker service:
$ sudo systemctl start docker.service

## Then we'll verify docker is installed successfully by checking the version:
$ docker version

Step 2: Install Docker Compose

## Install Extra Packages for Linux
$ sudo yum install epel-release

## Install python-pip
$ sudo yum install -y python-pip

## Then install Docker Compose:
$ sudo pip install docker-compose

## You will also need to upgrade your Python packages on CentOS 7 to get docker-compose to run successfully:
$ sudo yum upgrade python*

## To verify the successful Docker Compose installation, run:
$ docker-compose version

On Ubuntu 16.04+

Step 1: Install Docker

## Update the apt package index:
$ sudo apt-get update

## Install necessary packages:
$ sudo apt-get install  -y systemd apt-transport-https ca-certificates curl gnupg-agent software-properties-common 

## Add Docker's official GPG key:
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

## Add the apt repository
$ add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

## Update the apt package index:
$ sudo apt-get update

## Install docker-ce:
$ sudo apt-get -y install docker-ce docker-ce-cli containerd.io

## Add your user to the docker group with the following command.
$ sudo usermod -aG docker $(whoami)

## Set Docker to start automatically at boot time:
$ sudo systemctl enable docker.service

## Finally, start the Docker service:
$ sudo systemctl start docker.service

## Then we'll verify docker is installed successfully by checking the version:
$ docker version

Step 2: Install Docker Compose

## Install python-pip
$ sudo apt-get install -y python-pip

## Then install Docker Compose:
$ sudo pip install docker-compose

## To verify the successful Docker Compose installation, run:
$ docker-compose version

Running a P-Rep Node on Docker Container

You have docker installed, then proceed the following steps to install the P-Rep node.

Step 1. Pull the docker image

Pull the latest stable version of an image.

$ docker pull iconloop/prep-node:1909301159xa7e757

Step 2. Run the P-Rep Node as a Docker container

Using docker command

$ docker run -d  -p 9000:9000 -p 7100:7100 -v ${PWD}/data:/data iconloop/prep-node:1909301159xa7e757

Using docker-compose command (Recommended)

Open docker-compose.yml in a text editor and add the following content:

version: '3'
services:
   prep:
      image: iconloop/prep-node:1909301159xa7e757
      container_name: "prep-node"
      network_mode: host
      restart: always
      environment:
         LOOPCHAIN_LOG_LEVEL: "DEBUG"
         ICON_LOG_LEVEL: "DEBUG"
         DEFAULT_PATH: "/data/loopchain"
         LOG_OUTPUT_TYPE: "file"
         CERT_PATH: "/cert"     
         iissCalculatePeriod: "1800"
         termPeriod: "1800"
         PRIVATE_PATH: "/cert/{==YOUR_KEYSTORE or YOUR_CERTKEY FILENAME==}"
         PRIVATE_PASSWORD: "{==YOUR_KEY_PASSWORD==}"
      cap_add:
         - SYS_TIME
      volumes:
         - ./data:/data
         - ./cert:/cert  
      ports:
         - 9000:9000
         - 7100:7100

Run docker-compose

$ docker-compose up -d

Above command options do the followings.

Map container ports 7100 and 9000 to the host ports.
Mount a volume into the docker container.
- -v ${PWD}/data:/data sets up a bind mount volume that links the /data/ directory from inside the P-Rep Node container to the ${PWD}/data directory on the host machine.
- data folder will have the following directory structure.

.
|---- data  
|     |---- PREP-TestNet   → Default ENV directory  
|          |---- .score_data  
|          |      |-- db      → root directory that SCOREs will be installed
|          |      |-- score   → root directory that the state DB file will be created
|          |---- .storage     → root directory that the block DB will be stored
|          |---- log          → root directory that log files will be stored

P-Rep Node Operation and Configuration

Start Node

Run docker-compose up.

$ docker-compose up -d
prep_prep_1 is up-to-date

The docker ps command shows the list of running docker containers.

$ docker ps
CONTAINER ID   IMAGE                                                          COMMAND                CREATED              STATUS                          PORTS                                                                 NAMES
0de99e33cdc9     iconloop/prep-node:1909301159xa7e757    "/src/entrypoint.sh"      2 minutes ago        Up 2 minutes(healthy)    0.0.0.0:7100->7100/tcp, 0.0.0.0:9000->9000/tcp prep_prep_1

The meaning of each column in the docker ps result output is as follows.

Column	Description
CONTAINER ID	Container ID
IMAGE	P-Rep Node's image name
COMMAND	The script will be executed whenever a P-Rep Node container is run
STATUS	Healthcheck status. One of "starting" , "healthy", "unhealthy" or "none"
PORTS	Exposed ports on the running container
NAMES	Container name

You can read the container booting log from the log folder.

$ tail -f data/PREP-TestNet/log/booting_20190419.log
[2019-09-30 02:19:01.454] DEFAULT_STORAGE_PATH=/data/PREP-TestNet/.storage
[2019-09-30 02:19:01.459] scoreRootPath=/data/PREP-TestNet/.score_data/score
[2019-09-30 02:19:01.464] stateDbRootPath=/data/PREP-TestNet/.score_data/db
[2019-09-30 02:19:01.468] P-REP package version info - 1909301159xa7e757
[2019-09-30 02:19:02.125] iconcommons             1.1.2
                          iconrpcserver           1.4.3
                          iconsdk                 1.2.0
                          iconservice             1.5.8
                          loopchain               2.4.5
[2019-09-30 02:19:07.107] Enable rabbitmq_management
[2019-09-30 02:19:10.676] Network: PREP-TestNet
[2019-09-30 02:19:10.682] Run loop-peer and loop-channel start
[2019-09-30 02:19:10.687] Run iconservice start!
[2019-09-30 02:19:10.692] Run iconrpcserver start!

Stop Node

$ docker-compose down

Stopping prep_prep_1 ... done
Removing prep_prep_1 ... done
Removing network prep_default

View Node Status

$ curl localhost:9000/api/v1/status/peer
{
  "made_block_count": 0,
  "leader_made_block_count": 1,
  "status": "Service is online: 0",
  "state": "Vote",
  "service_available": true,
  "peer_type": "0",
  "audience_count": "0",
  "consensus": "siever",
  "peer_id": "hxab7f0442e884affaeb72aa0399ad1f0c1bd7fe3f",
  "block_height": 1265,
  "round": 0,
  "epoch_height": 1266,
  "unconfirmed_block_height": 1266,
  "total_tx": 1374,
  "unconfirmed_tx": 0,
  "peer_target": "20.20.1.254:7100",
  "leader_complaint": 1,
  "peer_count": 14,
  "leader": "hx74cc5ad0e7f6a92a8db684a1694cbfe16e50bc6e",
  "epoch_leader": "hx74cc5ad0e7f6a92a8db684a1694cbfe16e50bc6e",
  "versions": {
    "loopchain": "2.4.5",
    "iconservice": "1.5.8",
    "iconrpcserver": "1.4.3",
    "iconcommons": "1.1.2",
    "earlgrey": "0.0.4"
  },
  "mq": {
    "peer": {
      "message_count": 0
    },
    "channel": {
      "message_count": 0
    },
    "score": {
      "message_count": 0
    }
  }
}

Docker Environment Variables

If you want change the TimeZone, open docker-compose.yml in a text editor and add the following content:

version: '3' services:    container:        image: 'iconloop/prep-node:1909301159xa7e757'        container_name: 'prep-node'        volumes:            - ./data:/data        ports:           - 9000:9000           - 7100:7100       environment:          TZ: "America/Los_Angeles"

The P-Rep Node image supports the following environment variables:

Environment variable	Description	Default value	Allowed value
IPADDR	Setting the IP address	$EXT_IPADDR
TZ	Setting the TimeZone Environment	Asia/Seoul	List of TZ name
NETWORK_ENV	Network Environment name	PREP-TestNet
SERVICE	Service Name	zicon
ENDPOINT_URL	ENDPOINT API URI	https://${SERVICE}.net.solidwallet.io	URI
SERVICE_API	SERVICE_API URI	${ENDPOINT_URL}/api/v3	URI
NTP_SERVER	NTP SERVER ADDRESS	time.google.com
NTP_REFRESH_TIME	NTP refresh time	21600
FIND_NEIGHBOR	Find fastest neighborhood PRrep	true
GENESIS_NODE	false	false
DEFAULT_PATH	Setting the Default Root PATH	/data/${NETWORK_ENV}
DEFAULT_LOG_PATH	Setting the logging path	${DEFAULT_PATH}/log
DEFAULT_STORAGE_PATH	block DB will be stored	${DEFAULT_PATH}/.storage
USE_NAT	if you want to use NAT Network	no
NETWORK_NAME
VIEW_CONFIG	for check deployment state	false	boolean (true/false)
AMQP_TARGET	127.0.0.1	127.0.0.1
USE_MQ_ADMIN	Enable RabbitMQ management Web interface.The management UI can be accessed using a Web browser at http://{node-hostname}:15672/. For example, for a node running on a machine with the hostname of prep-node, it can be accessed at http://prepnode:15672/	false	boolean (true/false)
MQ_ADMIN	RabbitMQ management username	admin
MQ_PASSWORD	RabbitMQ management password	iamicon
LOOPCHAIN_LOG_LEVEL	loopchain log level	INFO	DEBUG, INFO, WARNING, ERROR
ICON_LOG_LEVEL	iconservice log level	INFO	DEBUG, INFO, WARNING, ERROR
LOG_OUTPUT_TYPE	loopchain's output log type	file	file, console, file\|console
outputType	iconservice's output log type	$LOG_OUTPUT_TYPE	file, console, file\|console
FIRST_PEER	for testnet	false
NEWRELIC_LICENSE	for testnet
CONF_PATH	Setting the configure file path	/${APP_DIR}/conf
CERT_PATH	Setting the certificate key file path	/${APP_DIR}/cert
REDIRECT_PROTOCOL	http	http
SUBSCRIBE_USE_HTTPS	false	false
ICON_NID	Setting the ICON Network ID number	0x50
ALLOW_MAKE_EMPTY_BLOCK	true	true
score_fee	true	true
score_audit	false	false
scoreRootPath	${DEFAULT_PATH}/.score_data/score	${DEFAULT_PATH}/.score_data/score
stateDbRootPath	${DEFAULT_PATH}/.score_data/db	${DEFAULT_PATH}/.score_data/db
iissDbRootPath	${DEFAULT_PATH}/.iissDb	${DEFAULT_PATH}/.iissDb
CHANNEL_BUILTIN	boolean (true/false)	true
PEER_NAME	`uname`	`uname`
PRIVATE_PATH	private cert key or keystore file location	${CERT_PATH}/${IPADDR}_private.der
PRIVATE_PASSWORD	private cert key or keystore file password	test
LOAD_PEERS_FROM_IISS	true	true
CHANNEL_MANAGE_DATA_PATH	${CONF_PATH}/channel_manange_data.json	${CONF_PATH}/channel_manange_data.json
CONFIG_API_SERVER	https://download.solidwallet.io	https://download.solidwallet.io
GENESIS_DATA_PATH	${CONF_PATH}/genesis.json	${CONF_PATH}/genesis.json
BLOCK_VERSIONS
NEXT_BLOCK_VERSION_HEIGHT
FORCE_RUN_MODE	Setting the loopchain running parameter e.g. if FORCE_RUN_MODE is `-r citizen` then loop `-r citizen`
configure_json	${CONF_PATH}/configure.json	${CONF_PATH}/configure.json
iconservice_json	${CONF_PATH}/iconservice.json	${CONF_PATH}/iconservice.json
iconrpcserver_json	${CONF_PATH}/iconrpcserver.json	${CONF_PATH}/iconrpcserver.json
ICON_REVISION	5	5
ROLE_SWITCH_BLOCK_HEIGHT	1	1
mainPRepCount	22	22
mainAndSubPRepCount	100	100
decentralizeTrigger	0.002	0.002
iissCalculatePeriod	origin value is 43200	1800
termPeriod	origin value is 43120	1800
blockValidationPenaltyThreshold	66000000	66000000
lowProductivityPenaltyThreshold	85	85
RPC_PORT	Choose a RPC service port	9000
RPC_WORKER	Setting the number of RPC workers	3
RPC_GRACEFUL_TIMEOUT	rpc graceful timeout	0
USE_PROC_HEALTH_CHECK	yes	yes
USE_API_HEALTH_CHEK	yes	yes
USE_HELL_CHEK	yes	yes
HEALTH_CHECK_INTERVAL	Trigger if greater than 1	20
ERROR_LIMIT	3	3
HELL_LIMIT	30	30
USE_SLACK	if you want to use the slack	no
SLACK_URL	slack's webhook URL
SLACK_PREFIX	slack's prefix header message
CURL_OPTION	default curl options	-s -S --fail --max-time 30

Troubleshooting

Q: How to check if container is running or not

The docker ps command shows the list of running docker containers.

$ docker ps
CONTAINER ID   IMAGE                                                          COMMAND                CREATED              STATUS                          PORTS                                                                 NAMES
0de99e33cdc9     iconloop/prep-node:1909301159xa7e757    "/src/entrypoint.sh"      2 minutes ago        Up 2 minutes(healthy)    0.0.0.0:7100->7100/tcp, 0.0.0.0:9000->9000/tcp prep_prep_1

You should look at the STATUS field to see if the container is running up and in healthy state.

Inside the container, there is a healthcheck script running with the following configuration. It will return unhealthy when fails.

Healthcheck option	value
retries	4
interval	30s
timeout	20s
start-period	60s

The container can have three states:

starting - container just starts
healthy - when the health check passes
unhealthy - when the health check fails

If the container does not start properly or went down unexpectedly, please check the booting.log. Below is the log messages on success.

$ cat data/PREP-TestNet/log/booting_${DATE}.log 

[2019-09-30 02:19:01.435] Your IP: 20.20.1.195
[2019-09-30 02:19:01.439] RPC_PORT: 9000 / RPC_WORKER: 3
[2019-09-30 02:19:01.444] DEFAULT_PATH=/data/PREP-TestNet in Docker Container
[2019-09-30 02:19:01.449] DEFAULT_LOG_PATH=/data/PREP-TestNet/log
[2019-09-30 02:19:01.454] DEFAULT_STORAGE_PATH=/data/PREP-TestNet/.storage
[2019-09-30 02:19:01.459] scoreRootPath=/data/PREP-TestNet/.score_data/score
[2019-09-30 02:19:01.464] stateDbRootPath=/data/PREP-TestNet/.score_data/db
[2019-09-30 02:19:01.468] P-REP package version info - 1909301159xa7e757
[2019-09-30 02:19:02.125] iconcommons             1.1.2
                          iconrpcserver           1.4.3
                          iconsdk                 1.2.0
                          iconservice             1.5.8
                          loopchain               2.4.5
[2019-09-30 02:19:07.107] Enable rabbitmq_management
[2019-09-30 02:19:10.676] Network: PREP-TestNet
[2019-09-30 02:19:10.682] Run loop-peer and loop-channel start
[2019-09-30 02:19:10.687] Run iconservice start!
[2019-09-30 02:19:10.692] Run iconrpcserver start!

Q: How to find error

Error log messages example

Grep the ERROR messages from the log files to find the possible cause of the failure.

$ cat data/PREP-TestNet/log/booting_${DATE}.log | grep ERROR

[2019-09-30 02:08:48.746] [ERROR] Download Failed - http://20.20.1.149:5000/cert/20.20.1.195_public.der status_code=000

[2019-09-30 01:58:46.439] [ERROR] Unauthorized IP address, Please contact our support team

Docker container generates below log files

booting.log
- The log file contains the errors that occurred while the docker container starts up.
iconrpcserver.log
- The log file contains information about the request/response message handling going through the iconrpcserver.
iconservice.log
- The log file contains the internals of ICON Service
loopchain.channel-txcreator-icon_dex_broadcast.icon_dex.log
- The log file contains information about TX broadcast from a node to other nodes
loopchain.channel-txcreator.icon_dex.log
- The log file contains information about the process of confirming TX
loopchain.channel-txreceiver.icon_dex.log
- The log file contains information about receiving the broadcasted TX from a node.
loopchain.channel.icon_dex.log
- The log file contains information about internals of loopchain engine

Q: How to monitor resources

We recommend the following tools for resource monitoring

Network monitoring - iftop, nethogs, vnstat
CPU/Memory monitoring - top, htop
Disk I/O monitoring - iostat, iotop
Docker monitoring - docker stats, ctop

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

p-rep-installation-and-configuration-for-testnet.md

p-rep-installation-and-configuration-for-testnet.md

Intended Audience

Pre-requisites

HW Requirements for Testnet

SW Requirements

OS requirements

Package requirements

Network Diagram of P-Rep nodes

Inside a P-Rep Node

P-Rep Installation using Docker

Prerequisites - Docker & Docker Compose Installation

On Centos 7

On Ubuntu 16.04+

Running a P-Rep Node on Docker Container

Step 1. Pull the docker image

Step 2. Run the P-Rep Node as a Docker container

P-Rep Node Operation and Configuration

Start Node

Stop Node

View Node Status

Docker Environment Variables

Troubleshooting

Q: How to check if container is running or not

Q: How to find error

Q: How to monitor resources

Files

p-rep-installation-and-configuration-for-testnet.md

Latest commit

History

p-rep-installation-and-configuration-for-testnet.md

File metadata and controls

Intended Audience

Pre-requisites

HW Requirements for Testnet

SW Requirements

OS requirements

Package requirements

Network Diagram of P-Rep nodes

Inside a P-Rep Node

P-Rep Installation using Docker

Prerequisites - Docker & Docker Compose Installation

On Centos 7

On Ubuntu 16.04+

Running a P-Rep Node on Docker Container

Step 1. Pull the docker image

Step 2. Run the P-Rep Node as a Docker container

P-Rep Node Operation and Configuration

Start Node

Stop Node

View Node Status

Docker Environment Variables

Troubleshooting

Q: How to check if container is running or not

Q: How to find error

Q: How to monitor resources