Despite the fact that on HabrΓ© there was already
not one article about IPFS .
Iβll clarify right away that Iβm not an expert in this field, but I have shown interest in this technology more than once, but trying to play with it often caused some pain. Today I again took up the experiments and got some results that I would like to share. In short, the process of installing IPFS and some chips will be described (everything was performed on ubuntu, on other platforms I did not try).
If you missed what IPFS is, it is written in some detail here: habr.com/en/post/314768
Installation
For the purity of the experiment, I suggest immediately installing it on some external server, since we will consider some of the pitfalls with working in local mode and remote. Then, if desired, it will not be demolished for long, there are not many.
Set go
Official documentation
See the current version at
golang.org/dl
Note: it is better to install IPFS on behalf of the user, who is supposed to use the most frequent ones. The fact is that below we will consider the option of mounting via FUSE and there are subtleties.
cd ~ curl -O https://dl.google.com/go/go1.12.9.linux-amd64.tar.gz tar xvf go1.12.9.linux-amd64.tar.gz sudo chown -R root:root ./go sudo mv go /usr/local rm go1.12.9.linux-amd64.tar.gz
Then you need to update the environment (more details here:
golang.org/doc/code.html#GOPATH ).
echo 'export GOPATH=$HOME/work' >> ~/.bashrc echo 'export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin' >> ~/.bashrc source ~/.bashrc
Check that go is installed
go version
Install IPFS
I liked the way to install via
ipfs-update the most .
Install it with the command
go get -v -u github.com/ipfs/ipfs-update
After that, you can execute the following commands:
ipfs-update versions - to see all available versions for download.
ipfs-update version - to see the current installed version (until we have IPFS installed, it will be none).
ipfs-update install latest - install the latest version of IPFS. Instead of latest, respectively, you can specify any desired version from the list of available.
Install ipfs
ipfs-update install latest
Check
ipfs --version
Directly with the installation in general terms, everything.
IPFS launch
Initialization
First you need to perform initialization.
ipfs init
In response, you will get something like this:
ipfs init initializing IPFS node at /home/USERNAME/.ipfs generating 2048-bit RSA keypair...done peer identity: QmeCWX1DD7HnXXXXXXXXXXXXXXXXXXXXXXXXxxx to get started, enter: ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme
You can run the proposed command
ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme
Result Hello and Welcome to IPFS! ββββββββββ ββββββββββββββββ βββββββββββββββββββββββββββ βββββββββββββββββ ββββββββ ββββββββββ ββββββ ββββββββ ββββββ βββ ββββββββ ββββββ βββ ββββββββ If you're seeing this, you have successfully installed IPFS and are now interfacing with the ipfs merkledag! ------------------------------------------------------- | Warning: | | This is alpha software. Use at your own discretion! | | Much is missing or lacking polish. There are bugs. | | Not yet secure. Read the security notes for more. | ------------------------------------------------------- Check out some of the other files in this directory: ./about ./help ./quick-start <-- usage examples ./readme <-- this file ./security-notes
Here, in my opinion, the interesting is already starting. At the installation stage, the guys are already starting to use their own technologies. The proposed hash QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv - not generated specifically for you, but sewn into the release. That is, before the release, they prepared a welcome text, poured it into IPFS and added the address to the installer. In my opinion, this is very cool. And this file (more precisely, the entire folder) can now be viewed not only locally, but also on the official
ipfs.io/ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv gateway. At the same time, you can be sure that the contents of the folder did not change in any way, because if it had changed, then the hash would also have changed.
By the way, in this case, IPFS has some similarities with the version control server. If you make changes to the source files of the folder and again pour the folder into IPFS, then it will receive a new address. At the same time, the old folder will not go anywhere just like that and will be available at its former address.
Direct launch
ipfs daemon
They should get a response like this:
ipfs daemon Initializing daemon... go-ipfs version: 0.4.22- Repo version: 7 System version: amd64/linux Golang version: go1.12.7 Swarm listening on /ip4/xxxx/tcp/4001 Swarm listening on /ip4/127.0.0.1/tcp/4001 Swarm listening on /ip6/::1/tcp/4001 Swarm listening on /p2p-circuit Swarm announcing /ip4/127.0.0.1/tcp/4001 Swarm announcing /ip6/::1/tcp/4001 API server listening on /ip4/127.0.0.1/tcp/5001 WebUI: http://127.0.0.1:5001/webui Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080 Daemon is ready
We open the door to the Internet
Pay attention to these two lines:
WebUI: http://127.0.0.1:5001/webui Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Now, if you installed IPFS locally, then you will access IPFS interfaces at local addresses and everything will be available to you (For example,
localhost : 5001 / webui /). But when installed on an external server, by default, the gateways are closed to the Internet. There are two gateways:
- Admin webui ( github ) on port 5001.
- External API on port 8080 (readonly).
So far, for experimentation, you can open both ports (5001 and 8080), but on the battle server, of course, port 5001 should be closed with a firewall. There is 4001 ports, it is needed so that other peers can find you. It should be left open to outside requests.
Open ~ / .ipfs / config for editing and find these lines in it:
"Addresses": { "Swarm": [ "/ip4/0.0.0.0/tcp/4001", "/ip6/::/tcp/4001" ], "Announce": [], "NoAnnounce": [], "API": "/ip4/127.0.0.1/tcp/5001", "Gateway": "/ip4/127.0.0.1/tcp/8080" }
Change 127.0.0.1 to the ip of your server and save the file, then restart ipfs (stop the running command Ctrl + C and run again).
Must get
... WebUI: http://ip__:5001/webui Gateway (readonly) server listening on /ip4/ip__/tcp/8080
Now external interfaces should be available.
Check out
http://__ip_:8080/ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme
The above readme file should open.
http://__ip_:5001/webui/
The web interface should open.
If webui works for you, then IPFS settings can be changed directly in it, including viewing statistics, but below I will consider configuration options directly through the config file, which is generally not critical. Itβs just better to remember exactly where the config lies and what to do with it, and if the web face does not work, it will be more difficult.
Set up a web interface for working with your server
Here is the first pitfall, which was spent about three hours.
If you installed IPFS on an external server, but did not install or start IPFS locally, then when you access / webui in the web interface, you should see a connection error:
The fact is that webui, in my opinion, works very ambiguously. First, he tries to connect to the API of the server where the interface is open (of course, based on the address in the browser). and if it doesnβt work there, then it tries to connect to the local gateway. And if you have IPFS running locally, then your webui will work fine, only you will work with local IPFS, not external, although you opened webui on an external server. Then upload the files, but for some reason you donβt see them just like that on an external server ...
And if it is not running locally, we get a connection error. In our case, the error is most likely due to CORS, which is also indicated by webui, proposing to add a config.
ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["http://ip_ :5001", "http://127.0.0.1:5001", "https://webui.ipfs.io"]' ipfs config --json API.HTTPHeaders.Access-Control-Allow-Methods '["PUT", "GET", "POST"]'
I just registered a wildcard
ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["*"]'
Added headers can be found all in the same ~ / .ipfs / config. In my case it is
"API": { "HTTPHeaders": { "Access-Control-Allow-Origin": [ "*" ] } },
We restart ipfs and see that webui has successfully connected (in any case, it should if you opened the gateways for requests from outside, as described above).
Now you can directly upload folders and files through the web interface, as well as create your own folders.
Mount the FUSE file system
This is a pretty interesting trick.
We can add files (as well as folders) not only through the web interface, but also directly in the terminal, for example
ipfs add test -r added QmfYuz2gegRZNkDUDVLNa5DXzKmxxxxxxxxxx test/test.txt added QmbnzgRVAP4fL814h5mQttyqk1aURxxxxxxxxxxxx test
The last hash is the root folder hash.
Using this hash, we can open the folder on any ipfs node (which can find our node and get the contents), we can in the web interface on port 5001 or 8080, or we can locally through ipfs.
ipfs ls QmbnzgRVAP4fL814h5mQttyqk1aUxxxxxxxxxxxxx QmfYuz2gegRZNkDUDVLNa5DXzKmKVxxxxxxxxxxxxxx 10 test.txt
But you can still open it like a normal folder.
Let's create two folders in the root and give them rights to our user.
sudo mkdir /ipfs /ipns sudo chown USERNAME /ipfs /ipns
and restart ipfs with the --mount flag
ipfs daemon --mount
You can create folders in other places and specify the path to them via ipfs daemon --mount --mount-ipfs / ipfs_path --mount-ipns / ipns_path
Now reading from this folder is somewhat unusual.
ls -la /ipfs ls: reading directory '/ipfs': Operation not permitted total 0
That is, there is no direct access to the root of this folder. But then you can get the contents, knowing the hash.
ls -la /ipfs/QmbnzgRVAP4fL814h5mQttyqxxxxxxxxxxxxxxxxx total 0 -r--r--r-- 1 root root 10 Aug 31 07:03 test.txt
cat /ipfs/QmbnzgRVAP4fL814h5mQttyqxxxxxxxxxxxxxxxxx/test.txt test test
Moreover, even autocompletion inside the folder works when specifying a path.
As I said above, there are subtleties with such mounting: by default, mounted FUSE folders are accessible only to the current user (even root cannot read from this folder, not to mention other users in the system). If you want to make these folders available to other users, then in the config you need to change βFuseAllowOtherβ: false to βFuseAllowOtherβ: true. But that is not all. If you run IPFS as root, then everything is OK. And if on behalf of a regular user (albeit sudo), you get an error
mount helper error: fusermount: option allow_other only allowed if 'user_allow_other' is set in /etc/fuse.conf
In this case, you need to fix /etc/fuse.conf by uncommenting the line #user_allow_other.
After that, restart ipfs.
Known issues with FUSE
More than once the problem was noticed that after restarting ipfs with mounting (or maybe in other cases), the mount points / ipfs and / ipns become unavailable. There is no access to them, but ls -la / ipfs shows ???? in the list of rights.
Found such a solution:
fusermount -z -u /ipfs fusermount -z -u /ipns
Then restart ipfs.
Add service
Of course, launching in the terminal is only suitable for primary tests. In combat mode, the daemon should start automatically when the system starts.
On behalf of sudo, create the file /etc/systemd/system/ipfs.service and write to it:
[Unit] Description=IPFS Daemon After=syslog.target network.target remote-fs.target nss-lookup.target [Service] Type=simple ExecStart=/home/USERNAME/work/bin/ipfs daemon --mount User=USERNAME Restart=always [Install] WantedBy=multi-user.target
USERNAME, of course, must be replaced with your user (and perhaps the full path to the ipfs program will be different for you (you must specify the full path)).
We activate the service.
sudo systemctl enable ipfs.service
We start the service.
sudo service ipfs start
Check the status of the service.
sudo service ipfs status
For the purity of the experiment, it will be possible to reboot the server in the future to verify that ipfs successfully starts automatically.
Add the feasts we know
Consider the situation when we have IPFS nodes installed both on an external server and locally. On an external server, we add some kind of file and try to get it through IPFS locally by CID. What will happen? Of course, the local server most likely does not know anything about our external server and will simply try to find the file by CID by βaskingβ all IPFS peers available to it (with which it has already managed to βget to knowβ). Those, in turn, will ask others. And so, until the file is found. Actually, the same thing happens when we try to get the file through the official
ipfs.io gateway. If you're lucky, then the file will be found in a few seconds. And if not, it will not be found in a few minutes, which greatly affects the comfort of work. But we know where this file will first appear. So why donβt we immediately tell our local server βLook there firstβ? Apparently, this can be done.
1. We go to the remote server and look for ~ / .ipfs / config in the config
"Identity": { "PeerID": "QmeCWX1DD7HnPSuMHZSh6tFuxxxxxxxxxxxxxxxx",
2. We execute sudo service ipfs status and look for Swarm entries in it, for example:
Swarm announcing /ip4/ip__/tcp/4001
3. From this we add the general address of the form "/ ip4 / ip_your_server / tcp / 4001 / ipfs / $ PeerID".
4. For reliability, through our local webui, we will try to add this address to peers.
5. If everything is OK, open the local config ~ / .ipfs / config, find βBootstrapβ in it: [...
and first add the received address to the array.
Restart IPFS.
Now add the file to the external server and try to request it on the local one. Must fly fast.
But this functionality is still unstable. As far as I understand, even if we specify the peer address in Bootstrap, in the process of operation ipfs changes the list of active connections with peers. In any case, a discussion of this and wishes on the possibility of indicating constant peers is being held
here and it seems like
it is supposed to add some functionality to ipfs@5.0+
A list of current peers can be viewed both in webui and in the terminal.
ipfs swarm peers
Both there and there you can add your feast manually.
ipfs swarm connect "/ip4/ip__/tcp/4001/ipfs/$PeerID"
Until you have improved this functionality, you can write a tool to check for a connection with the desired peer, and if not, to add a connection.
Reasoning
Among those already familiar with IPFS, there are both arguments for and against IPFS. In principle, the day before yesterdayβs
discussion prompted me to dig IPFS again. And with regards to the discussion mentioned above: I canβt say that I strongly oppose some of the above arguments expressed (I disagree only with the fact that one and a half programmers use IPFS). In general, both of them are right in their own way (especially
commentary on checks makes you think). But if you cast aside the moral and legal assessment, who will give a technical assessment of this technology? Personally, I have some kind of inner feeling that βit needs to be definitely, it has certain prospectsβ. But why exactly, there is no clear wording. Like if you look at the existing centralized tools, then in many respects they are very ahead (stability of work, speed of work, controllability, etc.). Nevertheless, I have one thought, which seems to make sense and which can hardly be implemented without such decentralized systems. Of course, I really wave my mind, but I would formulate it this way: the principle of disseminating information on the Internet needs to be changed.
I will explain. If you think so, now our information is disseminated on the principle "I hope that the person to whom I transferred it will protect it and it will not be lost or received to those to whom it was not intended." For example, it is easy to consider various mail services, cloud storage, etc. And what do we have in the end? On HabrΓ© Hub,
Information Security is on the first line and almost every day we receive news about another global leak. In principle, all the most interesting is listed in the <irony> wonderful </ irony> article
Summer is almost over. Almost no data leaked . That is, the main Internet giants are becoming larger, they are accumulating more and more information, and such leaks are a kind of informational atomic explosions. This has never happened, and here again. At the same time, although many understand that there are risks, they will continue to trust their data with third-party companies. Firstly, there is no particular alternative, and secondly, they promise that they have patched all the holes and that this will never happen again.
What option do I see? It seems to me that the data should initially be distributed openly. But openness in this case does not mean that everything should be easy to read. I am talking about the openness of storage and distribution, but not the total openness in reading. I assume that information should be distributed with public keys. After all, the principle of public / private keys is already old, almost like the Internet. If the information is not confidential and designed for a wide range, then it is laid out immediately with the public key (but still in encrypted form, just anyone can decrypt it with the existing key). And if not, then it is laid out without a public key, and the key itself is transmitted to that which must have access to this information. At the same time, the one who needs to read it should have only a key, and where to get this information, it should not really hover - he simply pulls it from the network (this is a new principle of distribution to the content, not to the address).
Thus, attackers for a mass attack will need to get a huge number of private keys, and it is unlikely that they can do this in one place. This task, as I see it, is more difficult than hacking a particular service.
And here another problem closes: proof of authorship. Now on the Internet you can find many quotes written by our friends. But where is the guarantee that they wrote them? Now, if each such record was accompanied by a digital signature, it would be much simpler. It doesnβt matter where this information lies, the main thing is the signature, which is obviously difficult to fake.
And here's what is interesting: IPFS already carries encryption tools (after all, it is built on blockchain technology). The private key is immediately indicated in the config.
"Identity": { "PeerID": "QmeCWX1DD7HnPSuMHZSh6tFuMxxxxxxxxxxxxxx", "PrivKey": "CAASqAkwggSkAgEAAoIBAQClZedVmj8JkPvT92sGrNIQmofVF3ne8xSWZIGqkm+t9IHNN+/NDI51jA0MRzpBviM3o/c/Nuz30wo95vWToNyWzJlyAISXnUHxnVhvpeJAbaeggQRcFxO9ujO9DH61aqgN1m+JoEplHjtc4KS5 pUEDqamve+xAJO8BWt/LgeRKA70JN4hlsRSghRqNFFwjeuBkT1kB6tZsG3YmvAXJ0o2uye+y+7LMS7jKpwJNJBiFAa/Kuyu3W6PrdOe7SqrXfjOLHQ0uX1oYfcqFIKQsBNj/Fb+GJMiciJUZaAjgHoaZrrf2b/Eii3z0i+QIVG7OypXT3Z9JUS60 KKLfjtJ0nVLjAgMBAAECggEAZqSR5sbdffNSxN2TtsXDa3hq+WwjPp/908M10QQleH/3mcKv98FmGz65zjfZyHjV5C7GPp24e6elgHr3RhGbM55vT5dQscJu7SGng0of2bnzQCEw8nGD18dZWmYJsE4rUsMT3wXxhUU4s8/Zijgq27oLyxKNr9T7 2gxqPCI06VTfMiCL1wBBUP1wHdFmD/YLJwOjV/sVzbsl9HxqzgzlDtfMn/bJodcURFI1sf1e6WO+MyTc3.................
+ / NDI51jA0MRzpBviM3o / c / Nuz30wo95vWToNyWzJlyAISXnUHxnVhvpeJAbaeggQRcFxO9ujO9DH61aqgN1m + JoEplHjtc4KS5 "Identity": { "PeerID": "QmeCWX1DD7HnPSuMHZSh6tFuMxxxxxxxxxxxxxx", "PrivKey": "CAASqAkwggSkAgEAAoIBAQClZedVmj8JkPvT92sGrNIQmofVF3ne8xSWZIGqkm+t9IHNN+/NDI51jA0MRzpBviM3o/c/Nuz30wo95vWToNyWzJlyAISXnUHxnVhvpeJAbaeggQRcFxO9ujO9DH61aqgN1m+JoEplHjtc4KS5 pUEDqamve+xAJO8BWt/LgeRKA70JN4hlsRSghRqNFFwjeuBkT1kB6tZsG3YmvAXJ0o2uye+y+7LMS7jKpwJNJBiFAa/Kuyu3W6PrdOe7SqrXfjOLHQ0uX1oYfcqFIKQsBNj/Fb+GJMiciJUZaAjgHoaZrrf2b/Eii3z0i+QIVG7OypXT3Z9JUS60 KKLfjtJ0nVLjAgMBAAECggEAZqSR5sbdffNSxN2TtsXDa3hq+WwjPp/908M10QQleH/3mcKv98FmGz65zjfZyHjV5C7GPp24e6elgHr3RhGbM55vT5dQscJu7SGng0of2bnzQCEw8nGD18dZWmYJsE4rUsMT3wXxhUU4s8/Zijgq27oLyxKNr9T7 2gxqPCI06VTfMiCL1wBBUP1wHdFmD/YLJwOjV/sVzbsl9HxqzgzlDtfMn/bJodcURFI1sf1e6WO+MyTc3.................
7LMS7jKpwJNJBiFAa / Kuyu3W6PrdOe7SqrXfjOLHQ0uX1oYfcqFIKQsBNj / Fb + GJMiciJUZaAjgHoaZrrf2b / Eii3z0i + QIVG7OypXT3Z9JUS60 "Identity": { "PeerID": "QmeCWX1DD7HnPSuMHZSh6tFuMxxxxxxxxxxxxxx", "PrivKey": "CAASqAkwggSkAgEAAoIBAQClZedVmj8JkPvT92sGrNIQmofVF3ne8xSWZIGqkm+t9IHNN+/NDI51jA0MRzpBviM3o/c/Nuz30wo95vWToNyWzJlyAISXnUHxnVhvpeJAbaeggQRcFxO9ujO9DH61aqgN1m+JoEplHjtc4KS5 pUEDqamve+xAJO8BWt/LgeRKA70JN4hlsRSghRqNFFwjeuBkT1kB6tZsG3YmvAXJ0o2uye+y+7LMS7jKpwJNJBiFAa/Kuyu3W6PrdOe7SqrXfjOLHQ0uX1oYfcqFIKQsBNj/Fb+GJMiciJUZaAjgHoaZrrf2b/Eii3z0i+QIVG7OypXT3Z9JUS60 KKLfjtJ0nVLjAgMBAAECggEAZqSR5sbdffNSxN2TtsXDa3hq+WwjPp/908M10QQleH/3mcKv98FmGz65zjfZyHjV5C7GPp24e6elgHr3RhGbM55vT5dQscJu7SGng0of2bnzQCEw8nGD18dZWmYJsE4rUsMT3wXxhUU4s8/Zijgq27oLyxKNr9T7 2gxqPCI06VTfMiCL1wBBUP1wHdFmD/YLJwOjV/sVzbsl9HxqzgzlDtfMn/bJodcURFI1sf1e6WO+MyTc3.................
Zijgq27oLyxKNr9T7 "Identity": { "PeerID": "QmeCWX1DD7HnPSuMHZSh6tFuMxxxxxxxxxxxxxx", "PrivKey": "CAASqAkwggSkAgEAAoIBAQClZedVmj8JkPvT92sGrNIQmofVF3ne8xSWZIGqkm+t9IHNN+/NDI51jA0MRzpBviM3o/c/Nuz30wo95vWToNyWzJlyAISXnUHxnVhvpeJAbaeggQRcFxO9ujO9DH61aqgN1m+JoEplHjtc4KS5 pUEDqamve+xAJO8BWt/LgeRKA70JN4hlsRSghRqNFFwjeuBkT1kB6tZsG3YmvAXJ0o2uye+y+7LMS7jKpwJNJBiFAa/Kuyu3W6PrdOe7SqrXfjOLHQ0uX1oYfcqFIKQsBNj/Fb+GJMiciJUZaAjgHoaZrrf2b/Eii3z0i+QIVG7OypXT3Z9JUS60 KKLfjtJ0nVLjAgMBAAECggEAZqSR5sbdffNSxN2TtsXDa3hq+WwjPp/908M10QQleH/3mcKv98FmGz65zjfZyHjV5C7GPp24e6elgHr3RhGbM55vT5dQscJu7SGng0of2bnzQCEw8nGD18dZWmYJsE4rUsMT3wXxhUU4s8/Zijgq27oLyxKNr9T7 2gxqPCI06VTfMiCL1wBBUP1wHdFmD/YLJwOjV/sVzbsl9HxqzgzlDtfMn/bJodcURFI1sf1e6WO+MyTc3.................
I am not a security specialist and I canβt know exactly how to use it correctly, but it seems to me that these keys are used at the level of exchange between IPFS nodes. And also
js-ipfs and such example projects as
orbit-db , on which
orbit.chat works. That is, theoretically, each device (mobile and not only) can be easily equipped with its own encryption-decryption machines. In this case, it remains only for everyone to take care of preserving their private keys and everyone will be responsible for their own security, and not be hostage to another human factor in some super-popular Internet giant.