How does a decentralized messenger on the blockchain
At the beginning of 2017, we started creating a messenger on the blockchain [the name and link are in the profile] by discussing the advantages over classic P2P messengers.
years passed and we were able to confirm our concept: instant messenger applications for iOS, Web PWA, Windows, GNU / Linux, Mac OS and Android are now available.
Today we will tell you how the messenger is arranged on the blockchain and how client applications can work with its API.
We wanted the blockchain to solve the security and privacy issues of classic P2P messengers:
Of the blockchain buns, there is also close integration with the cryptocurrencies Ethereum, Dogecoin, Lisk, Dash, Bitcoin (this is still in progress) and the ability to send tokens in chats. We even made a built-in crypto-exchanger.
And then - how it all works.
Everyone is already used to the fact that transactions in the blockchain transfer tokens (coins) from one user to another. Like bitcoin. We have created a special type of transaction for sending messages.
To send a message in the messenger on the blockchain, you need to go through several stages:
Stages 1-3 and 7 are performed locally on the client, and 5-6 on the network nodes.
The message is encrypted with the private key of the sender and the public key of the recipient. We will take the public key from the network, but for this the recipient's account must be initialized, that is, have at least one transaction. You can use the REST request
, and when you download chats, the public keys of the interlocutors will already be available.
The messenger encrypts messages using the curve25519xsalsa20poly1305 algorithm ( NaCl Box ). Since the account contains Ed25519 keys, in order to form a box, the keys must first be converted to Curve25519 Diffie-Hellman.
Here is an example in JavaScript:
A transaction has the following general structure:
For the transaction-message,
is the most important - you need to place the message in it in the
object with the structure:
Messages are also divided into types. Essentially, the
parameter tells how to understand
. You can send just text, or you can send an object with interesting interests inside - for example, this is how the messenger makes transfers of cryptocurrencies in chat rooms.
As a result, we form the transaction:
In order for everyone to be sure of the authenticity of the sender and the recipient, in the time of sending and the contents of the message, the transaction is signed. A digital signature allows you to verify the authenticity of the transaction with a public key - a private key is not needed for this.
But the signature itself is just performed by the private key:
It can be seen from the diagram that we first hash the transaction with SHA-256, and then sign Ed25519 EdDSA and get the signature
, and the transaction identifier is part of the SHA-256 hash.
1 - We form a data block, including a message
2 - We consider SHA-256 from the data block
3 - We sign the transaction
Since the network is decentralized, any of the nodes with an open API will do. We make a POST request for the
endpoint:
In response, we get a transaction ID of type
A distributed consensus-based system of nodes determines the “reliability” of a message transaction. From whom and to whom, when, whether the message was replaced by another, and whether the time of sending was indicated correctly. This is a very important advantage of the blockchain - there is no central structure that is responsible for checks, and the sequence of messages and their contents cannot be faked.
First, one node checks the reliability, and then sends it to others - if most say that everything is in order, the transaction will be included in the next block of the chain - this is consensus.
The part of the node code that is responsible for checking can be viewed in GitHub - validator.js and verify.js . Yeah, the node runs on Node.js.
If consensus is reached, the transaction with our message will fall into the next block, along with other reliable transactions.
Blocks have a strict sequence, and each subsequent block is formed on the basis of hashes of previous blocks.
The bottom line is that our message is also included in this sequence and cannot be “rearranged”. If several messages fall into the block, their order will be determined by the
messages.
The messenger application retrieves transactions from the blockchain that are sent to the addressee. To do this, we made the
endpoint.
All transactions are available to everyone - you can receive encrypted messages. But only the recipient can decrypt with his private key and the public key of the sender:
Since messages are delivered in this way for about 5 seconds — this is the time a new network block appeared — we came up with a socket connection client-node and node-to-node. When a node receives a new transaction, it checks its validity and transfers it to other nodes. The transaction is available to messenger clients even before consensus and inclusion in the block. So we will deliver messages instantly, as well as the usual messengers.
To store the address book, we made KVS - Key-Value Storage is another type of transaction in which the
is encrypted not with NaCl-box, but with NaCl-secretbox . So the messenger stores other data.
File / image transfer and group chats still require a lot of work. Of course, in the tyap-bloop format, this can be fastened quickly, but we want to maintain the same level of privacy.
Yes, there is still work to be done - ideally, real privacy implies that users will not connect to public network nodes, but will raise their own. What do you think, how many percent of users do this? That's right, 0. Partially, we managed to solve this issue with the Tor version of the messenger.
We have proven that a messenger on the blockchain can exist. Earlier there was only one attempt in 2012 - bitmessage , which failed due to the long message delivery time, CPU load and lack of mobile applications.
And skepticism is connected with the fact that messengers on the blockchain are ahead of the time - people are not ready to take responsibility for their account on themselves, ownership of personal information is not yet a trend, and technology does not allow ensuring high speeds on the blockchain. Following will appear more technological analogues of our project. You will see.
2.5
years passed and we were able to confirm our concept: instant messenger applications for iOS, Web PWA, Windows, GNU / Linux, Mac OS and Android are now available.
Today we will tell you how the messenger is arranged on the blockchain and how client applications can work with its API.
We wanted the blockchain to solve the security and privacy issues of classic P2P messengers:
- One click to create an account - no phones and emails, no access to address books and geolocations.
- Interlocutors never establish direct connections, all communication goes through a distributed system of nodes. IP addresses of users are not accessible to each other.
- All messages are encrypted End-to-End curve25519xsalsa20poly1305. It seems that you will not surprise anyone, but we have open source code.
- MITM attack is excluded - each message is a transaction and is signed by Ed25519 EdDSA.
- The message falls into its block. The sequence and
timestamp
blocks cannot be fixed, and therefore the order of messages. - “I didn’t say this” will not work with messages on the blockchain.
- There is no central structure that does checks for the “authenticity” of a message. This is done by a consensus-based distributed node system, and it belongs to users.
- Impossibility of censorship - accounts cannot be blocked and messages deleted.
- The 2FA blockchain is an alternative to the hellish 2FA by SMS, which has broken a lot of health.
- The ability to get all your dialogs from any device at any time is the ability not to store dialogs locally at all.
- Message Delivery Confirmation. Not to the user's device, but to the network. In fact, this is a confirmation of the recipient's ability to read your message. This is a useful feature for sending critical notifications.
Of the blockchain buns, there is also close integration with the cryptocurrencies Ethereum, Dogecoin, Lisk, Dash, Bitcoin (this is still in progress) and the ability to send tokens in chats. We even made a built-in crypto-exchanger.
And then - how it all works.
Message is a transaction
Everyone is already used to the fact that transactions in the blockchain transfer tokens (coins) from one user to another. Like bitcoin. We have created a special type of transaction for sending messages.
To send a message in the messenger on the blockchain, you need to go through several stages:
- Encrypt message text
- Put ciphertext into transaction
- Sign transaction
- Send a transaction to any host
- The distributed system of nodes determines the “reliability” of the message
- If everything is OK, the transaction with the message is included in the next block.
- The recipient retrieves the message transaction and decrypts
Stages 1-3 and 7 are performed locally on the client, and 5-6 on the network nodes.
Message encryption
The message is encrypted with the private key of the sender and the public key of the recipient. We will take the public key from the network, but for this the recipient's account must be initialized, that is, have at least one transaction. You can use the REST request
GET /api/accounts/getPublicKey?address={ADAMANT address}
, and when you download chats, the public keys of the interlocutors will already be available.
The messenger encrypts messages using the curve25519xsalsa20poly1305 algorithm ( NaCl Box ). Since the account contains Ed25519 keys, in order to form a box, the keys must first be converted to Curve25519 Diffie-Hellman.
Here is an example in JavaScript:
/** * Encodes a text message for sending to ADM * @param {string} msg message to encode * @param {*} recipientPublicKey recipient's public key * @param {*} privateKey our private key * @returns {{message: string, nonce: string}} */ adamant.encodeMessage = function (msg, recipientPublicKey, privateKey) { const nonce = Buffer.allocUnsafe(24) sodium.randombytes(nonce) if (typeof recipientPublicKey === 'string') { recipientPublicKey = hexToBytes(recipientPublicKey) } const plainText = Buffer.from(msg) const DHPublicKey = ed2curve.convertPublicKey(recipientPublicKey) const DHSecretKey = ed2curve.convertSecretKey(privateKey) const encrypted = nacl.box(plainText, nonce, DHPublicKey, DHSecretKey) return { message: bytesToHex(encrypted), nonce: bytesToHex(nonce) } }
Formation of a transaction with a message
A transaction has the following general structure:
{ "id": "15161295239237781653", "height": 7585271, "blockId": "16391508373936326027", "type": 8, "block_timestamp": 45182260, "timestamp": 45182254, "senderPublicKey": "bd39cc708499ae91b937083463fce5e0668c2b37e78df28f69d132fce51d49ed", "senderId": "U16023712506749300952", "recipientId": "U17653312780572073341", "recipientPublicKey": "23d27f616e304ef2046a60b762683b8dabebe0d8fc26e5ecdb1d5f3d291dbe21", "amount": 204921300000000, "fee": 50000000, "signature": "3c8e551f60fedb81e52835c69e8b158eb1b8b3c89a04d3df5adc0d99017ffbcb06a7b16ad76d519f80df019c930960317a67e8d18ab1e85e575c9470000cf607", "signatures": [], "confirmations": 3660548, "asset": {} }
For the transaction-message,
asset
is the most important - you need to place the message in it in the
chat
object with the structure:
-
message
- save the encrypted message -
own_message
- nonce -
type
- message type
Messages are also divided into types. Essentially, the
type
parameter tells how to understand
message
. You can send just text, or you can send an object with interesting interests inside - for example, this is how the messenger makes transfers of cryptocurrencies in chat rooms.
As a result, we form the transaction:
{ "transaction": { "type": 8, "amount": 0, "senderId": "U12499126640447739963", "senderPublicKey": "e9cafb1e7b403c4cf247c94f73ee4cada367fcc130cb3888219a0ba0633230b6", "asset": { "chat": { "message": "cb682accceef92d7cddaaddb787d1184ab5428", "own_message": "e7d8f90ddf7d70efe359c3e4ecfb5ed3802297b248eacbd6", "type": 1 } }, "recipientId": "U15677078342684640219", "timestamp": 63228087, "signature": " " } }
Transaction Signature
In order for everyone to be sure of the authenticity of the sender and the recipient, in the time of sending and the contents of the message, the transaction is signed. A digital signature allows you to verify the authenticity of the transaction with a public key - a private key is not needed for this.
But the signature itself is just performed by the private key:
It can be seen from the diagram that we first hash the transaction with SHA-256, and then sign Ed25519 EdDSA and get the signature
signature
, and the transaction identifier is part of the SHA-256 hash.
Implementation Example:
1 - We form a data block, including a message
/** * Calls `getBytes` based on transaction type * @see privateTypes * @implements {ByteBuffer} * @param {transaction} trs * @param {boolean} skipSignature * @param {boolean} skipSecondSignature * @return {!Array} Contents as an ArrayBuffer. * @throws {error} If buffer fails. */ adamant.getBytes = function (transaction) { ... switch (transaction.type) { case constants.Transactions.SEND: break case constants.Transactions.CHAT_MESSAGE: assetBytes = this.chatGetBytes(transaction) assetSize = assetBytes.length break … default: alert('Not supported yet') } var bb = new ByteBuffer(1 + 4 + 32 + 8 + 8 + 64 + 64 + assetSize, true) bb.writeByte(transaction.type) bb.writeInt(transaction.timestamp) ... bb.flip() var arrayBuffer = new Uint8Array(bb.toArrayBuffer()) var buffer = [] for (var i = 0; i < arrayBuffer.length; i++) { buffer[i] = arrayBuffer[i] } return Buffer.from(buffer) }
2 - We consider SHA-256 from the data block
/** * Creates hash based on transaction bytes. * @implements {getBytes} * @implements {crypto.createHash} * @param {transaction} trs * @return {hash} sha256 crypto hash */ adamant.getHash = function (trs) { return crypto.createHash('sha256').update(this.getBytes(trs)).digest() }
3 - We sign the transaction
adamant.transactionSign = function (trs, keypair) { var hash = this.getHash(trs) return this.sign(hash, keypair).toString('hex') } /** * Creates a signature based on a hash and a keypair. * @implements {sodium} * @param {hash} hash * @param {keypair} keypair * @return {signature} signature */ adamant.sign = function (hash, keypair) { return sodium.crypto_sign_detached(hash, Buffer.from(keypair.privateKey, 'hex')) }
Sending a transaction with a message to a host
Since the network is decentralized, any of the nodes with an open API will do. We make a POST request for the
api/transactions
endpoint:
curl 'api/transactions' -X POST \ -d 'TX_DATA'
In response, we get a transaction ID of type
{ "success": true, "nodeTimestamp": 63228852, "transactionId": "6146865104403680934" }
Transaction Validation
A distributed consensus-based system of nodes determines the “reliability” of a message transaction. From whom and to whom, when, whether the message was replaced by another, and whether the time of sending was indicated correctly. This is a very important advantage of the blockchain - there is no central structure that is responsible for checks, and the sequence of messages and their contents cannot be faked.
First, one node checks the reliability, and then sends it to others - if most say that everything is in order, the transaction will be included in the next block of the chain - this is consensus.
The part of the node code that is responsible for checking can be viewed in GitHub - validator.js and verify.js . Yeah, the node runs on Node.js.
Include transaction with message in block
If consensus is reached, the transaction with our message will fall into the next block, along with other reliable transactions.
Blocks have a strict sequence, and each subsequent block is formed on the basis of hashes of previous blocks.
The bottom line is that our message is also included in this sequence and cannot be “rearranged”. If several messages fall into the block, their order will be determined by the
timestamp
messages.
Reading messages
The messenger application retrieves transactions from the blockchain that are sent to the addressee. To do this, we made the
api/chatrooms
endpoint.
All transactions are available to everyone - you can receive encrypted messages. But only the recipient can decrypt with his private key and the public key of the sender:
** * Decodes the incoming message * @param {any} msg encoded message * @param {string} senderPublicKey sender public key * @param {string} privateKey our private key * @param {any} nonce nonce * @returns {string} */ adamant.decodeMessage = function (msg, senderPublicKey, privateKey, nonce) { if (typeof msg === 'string') { msg = hexToBytes(msg) } if (typeof nonce === 'string') { nonce = hexToBytes(nonce) } if (typeof senderPublicKey === 'string') { senderPublicKey = hexToBytes(senderPublicKey) } if (typeof privateKey === 'string') { privateKey = hexToBytes(privateKey) } const DHPublicKey = ed2curve.convertPublicKey(senderPublicKey) const DHSecretKey = ed2curve.convertSecretKey(privateKey) const decrypted = nacl.box.open(msg, nonce, DHPublicKey, DHSecretKey) return decrypted ? decode(decrypted) : '' }
What else?
Since messages are delivered in this way for about 5 seconds — this is the time a new network block appeared — we came up with a socket connection client-node and node-to-node. When a node receives a new transaction, it checks its validity and transfers it to other nodes. The transaction is available to messenger clients even before consensus and inclusion in the block. So we will deliver messages instantly, as well as the usual messengers.
To store the address book, we made KVS - Key-Value Storage is another type of transaction in which the
asset
is encrypted not with NaCl-box, but with NaCl-secretbox . So the messenger stores other data.
File / image transfer and group chats still require a lot of work. Of course, in the tyap-bloop format, this can be fastened quickly, but we want to maintain the same level of privacy.
Yes, there is still work to be done - ideally, real privacy implies that users will not connect to public network nodes, but will raise their own. What do you think, how many percent of users do this? That's right, 0. Partially, we managed to solve this issue with the Tor version of the messenger.
We have proven that a messenger on the blockchain can exist. Earlier there was only one attempt in 2012 - bitmessage , which failed due to the long message delivery time, CPU load and lack of mobile applications.
And skepticism is connected with the fact that messengers on the blockchain are ahead of the time - people are not ready to take responsibility for their account on themselves, ownership of personal information is not yet a trend, and technology does not allow ensuring high speeds on the blockchain. Following will appear more technological analogues of our project. You will see.
All Articles