Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update /concepts/nodes page #1624

Merged
merged 7 commits into from
Jun 29, 2023
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 26 additions & 87 deletions docs/concepts/nodes.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
---
title: "Nodes"
description: "Participants in the IPFS network are called nodes. Nodes are the most important aspect of IPFS - without nodes running the IPFS daemon, there would be no IPFS network. This page discusses what nodes are, current IPFS implementations, and the types of services different nodes can offer."
description: "Participants in the IPFS network are called nodes. Nodes are the most important aspect of IPFS - without nodes running the IPFS daemon, there would be no IPFS network. This page discusses what nodes are, current IPFS implementations, and the types of services different nodes can offer."
sidebarDepth: 2
---

# Nodes

Participants in the IPFS network are called _nodes_. Nodes are an IPFS program that you run on your local computer to store files and connect to the IPFS network. They're the most crucial aspect of IPFS. Without nodes running the IPFS daemon (explained below), there would be no IPFS Network.
Participants in the IPFS network are called _nodes_. A _node_ is a program or process implementing IPFS that you run on your local computer (directly or via a browser) to store files and connect to the IPFS network. They're the most crucial aspect of IPFS. Without nodes running as an IPFS _daemon_ (explained below), there would be no IPFS Network.

You're likely to see the term _node_ throughout the IPFS docs, issues, and related code. It's a very general term, so its meaning depends on the context. There are three main categories of nodes: IPFS nodes, data nodes, and libp2p nodes for applications.

* __IPFS Nodes__ are programs that run on a computer that can exchange data with other IPFS nodes. They go by several different names, but we refer to them by a different term, depending on the context:
* _node_: Use _node_ when you're referring to an individual point on the network. It's a very general term. For example, when you open IPFS Desktop, you establish yourself as a node with the potential to interact with other nodes. See [Configure a node](https://docs.ipfs.tech/how-to/configure-node/).
* _node_: Use _node_ when you're referring to an individual point on the network. It's a very general term. For example, when you open IPFS Desktop, you establish yourself as a node with the potential to interact with other nodes. See [Configure a node](../how-to/configure-node.md).
* _peer_: Use _peer_ when you're talking about the relationship of one node (even your own) to other nodes. It refers to their relationship as equals, with no central authority, so your node is a peer to other peers. See [Observe peers](../how-to/observe-peers.md), [Exchange files between nodes](../how-to/exchange-files-between-nodes.md), and [Peering with content providers](../how-to/peering-with-content-providers.md).
* _daemon_: Use _daemon_ when talking about a node's activity status. When a node is online and running in the background, listening for requests for its data, it's called a _daemon_. See [Take your node online](../how-to/command-line-quick-start.md#take-your-node-online)
* _instance_: Use _instance_ when talking about a library or program, such as a Go or JS version, running on an IPFS node at a particular point in time. The peer ID is the same, so it's still the same _node_ as far as the IPFS network is concerned. See [Kubo](../reference/go/api.md) and [JS-IPFS](../reference/js/api.md#ipfs-and-javascript).
* _daemon_: Use _daemon_ when talking about a node's activity status. When a node is online and running in the background, listening for requests for its data, it's called a _daemon_. See [Take your node online](../how-to/command-line-quick-start.md#take-your-node-online). TODO_JS_IPFS_DEPRECATION: we should probably address how a Helia _node_ in the browser may not normally be referred to as a _daemon_, but in the context of this doc, we will refer to a Helia _instance_ acting as a _node_ in the browser would be defined as a _daemon_ for our purposes.
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
ElPaisano marked this conversation as resolved.
Show resolved Hide resolved
* _instance_: Use _instance_ when talking about a library or program, such as a Go or JS version, running on as an IPFS node at a particular point in time. The peer ID is the same, so it's still the same _node_ as far as the IPFS network is concerned. See [Kubo](../reference/go/api.md) and [Helia](../reference/js/api.md#TODO_JS_IPFS_DEPRECATION).
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved

* __Data nodes__, Use _data nodes_ when talking about actual pieces of data on IPFS, such as DAG nodes, UnixFS nodes, and IPLD nodes. When you add a file with the `ipfs add myfile.txt` command, IPFS breaks them up into several nodes that each contain a chunk of the file and are linked to each other. See [Merkle Directed Acyclic Graphs (DAGs)](../concepts/merkle-dag.md), [Unix File System (UnixFS)](../concepts/file-systems.md#unix-file-system-unixfs), and stay tuned for [InterPlanetary Linked Data (IPLD) model](../concepts/ipld.md) docs, which is in progress.

Expand All @@ -25,42 +25,24 @@ You're likely to see the term _node_ throughout the IPFS docs, issues, and relat

There are different types of IPFS nodes. And depending on the use-case, a single IPFS node can serve one of many functions:

- [Preload](#preload)
- [Relay](#relay)
- [Bootstrap](#bootstrap)
- [Delegate routing](#delegate-routing-node)

### Preload
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved

Use to make a UnixFS DAG publicly available by calling `ipfs refs -r <CID>` on a randomly chosen preload node's HTTP API. This puts the CID in the preload nodes' wantlist, causing it to fetch the data from the user. Other nodes requesting the content can then resolve it from the preload node using bitswap, as the data is now present in the preload node’s blockstore.

#### Features of a preload node

- They are Kubo nodes with API ports exposed. Some HTTP API commands are accessible.
- Used by JS-IPFS nodes running in browser contexts.
- JS-ipfs nodes remain connected to the libp2p swarm ports of all preload nodes by having preload nodes on the bootstrap list.
- Often on the same _server_ as a [delegate routing node](#delegate-routing-node), though both the delegate routing service and preload service are addressed differently. This is done by having different multiaddrs that resolve to the same machine.
- Preload nodes are in the default JS-IPFS configuration as bootstrap nodes, so they will maintain libp2p swarm connections to them at all times.
- They are configured as regular bootstrap nodes, but as a convention have the string 'preload' in their `/dnsaddr` multiaddrs.

#### Limitations of a preload node:

- Default preload nodes provided by Protocol Labs garbage collect every hour, so preloaded content only survives for that long. However, this is configurable. You can run nodes with customized policies.
- Requires client to be smart about what gets preloaded: recursive preload of a big DAG.
- Only works with dag-pb CIDs because that's all the refs command understands. It's harder to find non-dag-pb content, e.g., you need a connection to the publishing js-ipfs instance or it needs to be put on the DHT by a delegate node.

### Relay

If an IPFS node deems itself unreachable by the public internet, IPFS nodes may choose to use a relay node as a kind of VPN in an attempt to reach the unreachable node.

#### Features of a relay node

- Implements either [v1](https://github.com/libp2p/specs/blob/master/relay/circuit-v1.md) or [v2](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md) of the Circuit Relay protocol.
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
- Can be either Kubo or JS-IPFS nodes; however there are standalone implementations as well:
- [js-libp2p-relay-server](https://github.com/libp2p/js-libp2p-relay-server) (supports circuit v1)
- Can be either Kubo or Helia nodes; however there are standalone implementations as well:
- [js-libp2p/circuit-relay](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#setup-with-relay) (supports circuit v1 & v2) TODO_JS_IPFS_DEPRECATION - CONFIRM that js-libp2p still supports relay v1?
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
- [go-libp2p-relay-daemon](https://github.com/libp2p/go-libp2p-relay-daemon) (supports circuit v1 & v2)
- They're used by both Kubo and JS-IPFS nodes.
- JS-IPFS nodes can also use relay nodes to overcome the lack of transport compatibility within the JS-IPFS implementation. A browser node with WebSockets/webRTC transports can talk with a Kubo node that only communicates through TCP using a relay that supports both transports. This is not enabled by default and needs to be set up.
- They're used by both Kubo and Helia nodes.
- A Helia _node_ in the browser can't talk TCP, so a relay can help increase the number of _peers_ that can be communicated with. A Helia _node_ with browser-supported [transports](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#transport) can talk with a Kubo _peer_ with no overlapping transports using a relay _peer_ if that relay supports one transport from each other _peer_:
1. At least one browser-supported transport (appropriate transports are [enabled by default in Helia browser _nodes_](https://github.com/ipfs/helia/blob/d2a928aa1590d5aa642c4c6747d5282f665af43f/packages/helia/src/utils/libp2p-defaults.browser.ts))
2. At least one Kubo _peer_ supported transport.

#### Limitations of relay nodes:
- v1 relays can be used by anyone without any limits, unless [go-libp2p-relay-daemon](https://github.com/libp2p/go-libp2p-relay-daemon) is used with ACLs (Access Control Lists) set up.
Expand All @@ -71,82 +53,39 @@ See [p2p-circuit relay](https://github.com/libp2p/specs/tree/master/relay)

### Bootstrap

Both Kubo and JS-IPFS nodes use bootstrap nodes to initially enter the DHT.
Both Kubo and Helia _nodes_ use bootstrap _nodes_ to initially enter the DHT.

#### Features of a bootstrap node:

- All default bootstrap nodes are part of the public DHT.
- The list of bootstrap nodes a JS-IPFS node connects to is configurable in their config files.
- All default bootstrap _nodes_ are part of the public DHT.
- The list of bootstrap _nodes_ a Helia _node_ connects to is configurable in their config files.

#### Limitations of a bootstrap node:

- If an IPFS node only has one bootstrap node listed in that configuration and that bootstrap node goes offline, the IPFS node will lose access to the public DHT if it were to restart.
- If an IPFS _node_ only has one bootstrap _node_ listed in that configuration and that bootstrap node goes offline, the IPFS node will lose access to the public DHT if it were to restart.
- Note that you should be able to configure your _peer_ store in your implementation to cache healthy connectable _peers_ so you can connect to them again after a restart, instead of bootstrap _nodes_. [Kubo recently added support for this](https://github.com/ipfs/kubo/pull/8856).

[More about Bootstrapping](../how-to/modify-bootstrap-list.md)

### Delegate routing node

When IPFS nodes are unable to run Distributed Hash Table (DHT) logic on their own, they _delegate_ the task to a delegate routing node. Publishing works with arbitrary CID codecs (compression/decompression technology), as the [js-delegate-content module](https://github.com/libp2p/js-libp2p-delegated-content-routing/blob/master/src/index.ts) publishes CIDs at the block level rather than the IPLD or DAG level.
When IPFS nodes are unable to run Distributed Hash Table (DHT) logic on their own, they _delegate_ the task to a delegate routing node. Publishing works with arbitrary CID codecs (compression/decompression technology), as the [js-libp2p-delegated-content-routing module](https://github.com/libp2p/js-libp2p-delegated-content-routing/blob/master/src/index.ts) publishes CIDs at the block level rather than the IPLD or DAG level.

#### Features of a delegate routing node:

- They are Kubo nodes with their API ports exposed and some API commands accessible under `/api/v0`.
- Usable by both Kubo and JS-IPFS nodes.
- JS-IPFS nodes use them to query the DHT and also publish content without having to actually run DHT logic on their own.
- Often on the same _server_ as a [preload](#preload) node, though both the delegate routing service and preload service are addressed differently. This is done by having different multiaddrs that resolve to the same machine.
- Delegate routing nodes are in the default JS-IPFS configuration as bootstrap nodes, so they will maintain libp2p swarm connections to them at all times.
- They are configured as regular bootstrap nodes, but have the string 'preload' in their multiaddrs.
- They are IPFS _nodes_ with their API ports exposed and some [Kubo RPC API](../reference/kubo/rpc.md) commands accessible at path `/api/v0`.
- Usable by both Kubo and Helia _nodes_.
- Helia _nodes_ can use them to query the DHT and also publish content without having to actually run DHT logic on their own.
- The libp2p _node_ configured for a Helia _node_ can be configured for delegate routing by following instructions at in the [js-libp2p configuration doc](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#setup-with-content-and-peer-routing)

#### Limitations of a delegate routing node:

- On default delegate nodes provided by Protocol Labs, the garbage collection happens every hour, so provided content only survives for that long. If the uploading JS-IPFS node is still running, it will issue periodic re-provides using the same publishing mechanic, which extends the life of the content on the DHT.
- On default delegate _nodes_ provided by Protocol Labs, the garbage collection happens every hour, so provided content only survives for that long.
- Only Kubo is known to implement the [Kubo RPC API](../reference/kubo/rpc.md) at the time of writing.
- There is a successor to the delegated routing functionality of the Kubo RPC API called [Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/)

## Implementations

Protocol Labs manages two implementations of the IPFS spec: Kubo and JS-IPFS. These implementations use specific types of nodes to perform server, browser, and other client functions.

### Kubo

An implementation of IPFS in Go. Designed to run on servers and user machines with full IPFS capabilities, enabling experimentation. New IPFS features are usually created on Kubo before any other implementation.
More at [Github](https://github.com/ipfs/kubo#readme).

Features include:

- TCP and QUIC transports are enabled by default.
- `/ws/` transport disabled by default.
- HTTP gateway with subdomain support for origin isolation between content roots.
- Various [experimental features](https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md)

See [Working with Go](../reference/go/api.md#working-with-go) and [CLI Quick Start](../how-to/command-line-quick-start.md)

### JS-IPFS

:::warning
#### js-ipfs being discontinued
Development of the [js-ipfs project](https://github.com/ipfs/js-ipfs) is being discontinued to focus on [Helia](https://github.com/ipfs/helia), a leaner, more modular, modern implementation of IPFS in JavaScript scheduled for release in 2023. To learn more about Helia and the current state of IPFS in JS, see the [blog post](https://blog.ipfs.tech/state-of-ipfs-in-js/).

Because of this, js-ipfs tutorials may be out of date, and will eventually be archived.

:::

An implementation of IPFS written entirely in JavaScript. It runs in a browser, a Service Worker, Electron and Node.js. Capabilities depend on the runtime.

More at [js.ipfs.io](https://js.ipfs.io) and [Github](https://github.com/ipfs/js-ipfs#readme).

#### JS-IPFS node in a browser

When in a browser, runs with a limited set of IPFS capabilities.

Browser features include:

- Can connect to server nodes using secure WebSockets.
- WSS requires manual setup of TLS at the server.
- Can connect to a browser node using WebRTC using a centralized [ws-webrtc-star signaling service](https://github.com/libp2p/js-libp2p-webrtc-star).

Specific limitations of the JS-IPFS implementation are:

- Unless using WSS, a JS-IPFS node cannot connect to the main public DHT. They will only connect to other JS-IPFS nodes.
- The performance of the DHT is not on-par with the Kubo implementation.
- The HTTP gateway is present, but it has no subdomain support (can't open TCP port)
Protocol Labs manages two implementations of the IPFS spec: Kubo and Kubo. These implementations use specific types of _nodes_ to perform server, browser, and other client functions.
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved

More at [js.ipfs.io](https://js.ipfs.io) and [Github](https://github.com/ipfs/js-ipfs#readme).
[Read more about our implementations](./ipfs-implementations.md)