Dat - what kind of protocol is it and who uses it
We are talking about the principles of operation of this P2P protocol and projects built on its basis.
/ Unsplash / Alina Grubnyak
Dat (Decentralized Archive Transport) is an open protocol for exchanging data and messages within a distributed network (peer-to-peer). It has built-in version control functions and allows you to monitor changes in data sets. Dat was developed by engineer Max Ogden. He introduced the protocol in 2013 as part of the Code for America project.
Its participants develop services that contribute to the improvement of municipal services. Dat was created as a tool for transferring personal data of citizens between the state. organizations. But later, the focus of the project shifted towards scientific information.
A large community has formed around Dat (7 thousand stars on GitHub ). The protocol and the applications built on its basis are promoted by the nonprofit organization Dat Foundation . It is supported by Mozilla, the Code for Science & Society open source foundation and Wireline P2P developer.
To upload a file to the Dat network, you must specify its URL. Here is an example:
To find the address link, Dat clients use multicast DNS . Peers broadcast their request to the local network in the hope that one of the participants will “hear” and share information. Customers can also access the server on the Internet. The main server is located at discovery1.datprotocol.com . If it is not available, then you can refer to its mirror - discovery2.datprotocol.com .
When the peer learns the IP address and port number of another peer, they establish a TCP connection. All transmitted data is encrypted - for this, the XSalsa20 stream encryption system is used . The algorithm uses a twenty-cycle hash function. Conversion operations are reminiscent of those involved in AES.
Data in a Dat network is transmitted in separate fragments (chunks), the sizes of which may vary. The system allows you to add new fragments to the Dat file, but does not allow you to modify or delete existing ones. According to the developers, this approach allows you to save the entire history of document changes. The system is able to function freely in an environment with an unstable connection.
/ Unsplash / Sven Brandsma
Now members of the Dat Foundation are improving the protocol so that it can work with large amounts of data. In particular, they plan to rework the file system (called Hyperdrive ) to cope with millions of files and introduce new peer search engines ( Hyperswarm ).
An example would be an open ScienceFair project. This is a desktop application for browsing and searching for scientific literature. On this platform, scientists and researchers can work with personal notes, magazines or extracts from them. To display content from the scientific literature, ScienceFair uses a Lens reader - it is responsible for rendering the JATS XML format .
Although Dat was originally conceived as a protocol for exchanging scientific information, the community uses it to create websites, chats, and other applications.
One recent example is the Beaker P2P browser, which is developed in partnership with the Dat development team. Its purpose is to give users the ability to host websites "directly in the browser." Beaker authors launched the Hashbase cloud service, which provides constant access to dat sites whose local copies are not available.
The project is completely open, and its source can be found on GitHub . If you want to evaluate the capabilities of the browser yourself, then to run it on Linux you will need to install libtool, m4 and autoconf:
After enough run:
More sample applications can be found on the project website .
Of course, Dat is not the only P2P protocol that has been actively developed recently. An alternative is the IPFS (InterPlanetary File System) project. This is a peer-to-peer distributed file system. The difference from other decentralized networks is that it works with blocks. They can contain both a part of the file and links to other blocks. From these blocks a generalized tree-like directed graph is formed that forms a directory or file. IPFS works with distributed hash tables and supports decentralized block exchanges. IPFS has no points of failure, and hosts are not required to trust each other. Access to the file system can be obtained using FUSE or HTTP.
The solution already uses Neocities hosting and OpenBazaar marketplace. Developers of protocols like IPFS and Dat hope their projects will give Internet users more control over their data.
/ Unsplash / Alina Grubnyak
What is dat
Dat (Decentralized Archive Transport) is an open protocol for exchanging data and messages within a distributed network (peer-to-peer). It has built-in version control functions and allows you to monitor changes in data sets. Dat was developed by engineer Max Ogden. He introduced the protocol in 2013 as part of the Code for America project.
Its participants develop services that contribute to the improvement of municipal services. Dat was created as a tool for transferring personal data of citizens between the state. organizations. But later, the focus of the project shifted towards scientific information.
A large community has formed around Dat (7 thousand stars on GitHub ). The protocol and the applications built on its basis are promoted by the nonprofit organization Dat Foundation . It is supported by Mozilla, the Code for Science & Society open source foundation and Wireline P2P developer.
How does he work
To upload a file to the Dat network, you must specify its URL. Here is an example:
dat://778f8d955175c92e4ced5e4f5563f69bfec0c86cc6f670352c457943666fe639/dat_intro.gif
To find the address link, Dat clients use multicast DNS . Peers broadcast their request to the local network in the hope that one of the participants will “hear” and share information. Customers can also access the server on the Internet. The main server is located at discovery1.datprotocol.com . If it is not available, then you can refer to its mirror - discovery2.datprotocol.com .
About other protocols and standards in our blog on Habré:
When the peer learns the IP address and port number of another peer, they establish a TCP connection. All transmitted data is encrypted - for this, the XSalsa20 stream encryption system is used . The algorithm uses a twenty-cycle hash function. Conversion operations are reminiscent of those involved in AES.
Data in a Dat network is transmitted in separate fragments (chunks), the sizes of which may vary. The system allows you to add new fragments to the Dat file, but does not allow you to modify or delete existing ones. According to the developers, this approach allows you to save the entire history of document changes. The system is able to function freely in an environment with an unstable connection.
/ Unsplash / Sven Brandsma
Now members of the Dat Foundation are improving the protocol so that it can work with large amounts of data. In particular, they plan to rework the file system (called Hyperdrive ) to cope with millions of files and introduce new peer search engines ( Hyperswarm ).
Who uses
An example would be an open ScienceFair project. This is a desktop application for browsing and searching for scientific literature. On this platform, scientists and researchers can work with personal notes, magazines or extracts from them. To display content from the scientific literature, ScienceFair uses a Lens reader - it is responsible for rendering the JATS XML format .
Although Dat was originally conceived as a protocol for exchanging scientific information, the community uses it to create websites, chats, and other applications.
One recent example is the Beaker P2P browser, which is developed in partnership with the Dat development team. Its purpose is to give users the ability to host websites "directly in the browser." Beaker authors launched the Hashbase cloud service, which provides constant access to dat sites whose local copies are not available.
The project is completely open, and its source can be found on GitHub . If you want to evaluate the capabilities of the browser yourself, then to run it on Linux you will need to install libtool, m4 and autoconf:
sudo apt-get install libtool m4 make g++ autoconf # debian/ubuntu sudo dnf install libtool m4 make gcc-c++ libXScrnSaver # fedora
After enough run:
git clone https://github.com/beakerbrowser/beaker.git cd beaker npm install npm run rebuild # see https://github.com/electron/electron/issues/5851 npm start
More sample applications can be found on the project website .
Analogue
Of course, Dat is not the only P2P protocol that has been actively developed recently. An alternative is the IPFS (InterPlanetary File System) project. This is a peer-to-peer distributed file system. The difference from other decentralized networks is that it works with blocks. They can contain both a part of the file and links to other blocks. From these blocks a generalized tree-like directed graph is formed that forms a directory or file. IPFS works with distributed hash tables and supports decentralized block exchanges. IPFS has no points of failure, and hosts are not required to trust each other. Access to the file system can be obtained using FUSE or HTTP.
The solution already uses Neocities hosting and OpenBazaar marketplace. Developers of protocols like IPFS and Dat hope their projects will give Internet users more control over their data.
What we write about in the VAS Experts corporate blog:
All Articles