Introduction
There are many programs (TeamViewer, Hamachi) for remote computer control, data transfer, etc. They work surprisingly well and in any conditions. But as a rule, these are paid services. It is unknown how these programs work, and they don’t cause me much desire to use them. As a user of Linux (mainly Ubuntu), I used a bunch of OpenVPN, SSH, VNC, RDP and other services to transfer data and manage my PCs, depending on my needs. In order to organize a network between two computers, I used a VPN server so that these computers would connect to the OpenVPN server and all traffic went through the server. This is not very good, because traffic goes a double way, as a result of this, speed was lost.
Theory
I have long been thinking about a solution for organizing a data transmission channel directly from node to node, which are located behind NAT providers, without using intermediary servers. After going through a number of articles about various technologies such as GRE tunnels, IPsec, UDP Hole Punching , OpenVPN and other things, I realized that the nodes should punch the connection towards each other, that is, send packets to the IP and port of the remote node. I put several experiments on organizing a GRE tunnel through NAT, sending a message using NetCat towards each other, sometimes it worked, sometimes not, it all depended on the type of NAT used by the provider. Not so long ago an interesting article caught my eye, in which there was a description of the organization of the OpenVPN connection between two computers (hereinafter referred to as nodes). I read, checked and worked, but provided that the local port of the node and the external port match, that is, my provider will use Cone NAT. It became interesting for me to organize a tunnel between two computers, provided that both will be behind any type (Cone or Symmetric) NAT, that is, the local port may not coincide with the external port. The task rested in the impossibility of determining the current port of the external interface without external assistance. If you can somehow find out the external IP address (for example: curl ifconfig.me), but there is a problem with determining the current external port.
Practice
To solve this problem, I had to use VPS (S-blue oval), on it I raised a script that served as a “Connector”, something like a STUN server : using the TCPDump utility, I received a UDP packet (hereinafter the UDP protocol is used everywhere) on a specific interface and port, parse the contents of the packet, determine the IP address / port of the source and respond with the NetCat utility returning the current parameters (IP address and port) of the connection, as well as parameters (IP address and port) of the remote node to which you want to connect, if this data was available, otherwise I waited until it appears. All this business was compared with the identifier (ID) of the connection, since several connections could interfere with each other, when using the ID, everything was solved. There was also a problem that the node was receiving its own old data and trying to connect through it, and this was solved using the Hostname hash.
On nodes A and B, I used a script and a pre-generated key to authorize a VPN connection, it worked like this: a script was run, a local port was randomly selected in the range from 20,000 to 65,000, and a packet was sent from this port to the VPS, which contained the connection ID and the hostname hash, using the NetCat utility, and TCPDump was immediately launched waiting for a response. In response, a packet came that contained the current data (IP address / port) of this node, and if there was data from a remote node, they also came and the exchange of greetings between the nodes began. If there was no data of the remote node, then the poll was repeated at intervals of 30-45 seconds to maintain the session. At the moment when all the necessary data (the IP address and port of the current node and the remote one) were on the nodes, the packet exchange began, the packet contained the number m = 0 and the generated number from 0 to 254 (this number was used to generate the internal IP address VPN connections). Upon receiving a packet with m = 0, the remote node sent m = 1 in response, and so on up to 10. Upon receipt of the m = 10 packet, a packet packet was sent with a frequency of 1 second with m = 13 and OpenVPN started, while the remote node received m = 13 too launched OpenVPN using the local port, IP address and port of the remote host, as well as the generated internal IP of the form 10.XX {1,2} / 30.
Warning: scripts are written and tested on Ubuntu 18.04 and Debian 9
Script on VPS:
# cat connector2.sh
#!/bin/bash if [[ $1 == '' ]]; then echo -e " "; exit; fi iface=`ip route get 8.8.8.8 | head -n 1 | sed 's|.*dev ||' | awk '{print $1}'` a=0 until (( $a == 500)); do packet=`tcpdump -i $iface udp port $1 -vvn -c1 -A` if [[ "$packet" == *"Ident"* ]]; then pack=`echo "$packet" | grep -e "udp sum ok" -e "Ident"` myip=`echo $pack | sed 's/\./ /g' | awk '{print $1"."$2"."$3"."$4}'` myport=`echo $pack | sed 's/\./ /g' | awk '{print $5}'` id=`echo $pack | sed 's|.*Ident:||' | awk '{print $1}'` myname=`echo $pack | sed 's|.*Ident:||' | awk '{print $2}'` echo "$myip:$myport $myname" > /tmp/vpn2-$id-$myname echo "MyData $myip:$myport $(cat /tmp/vpn2-$id-* | grep -v $myname | awk {'print $1'})" | nc $myip $myport -u -p $1 -w 1 cat /tmp/vpn2-$id-* topoint=`cat /tmp/vpn2-$id-* | grep -v "$myname" | sed 's/:/ /g'` if [[ $topoint != '' ]]; then ip=`echo $topoint | awk '{print $1}'` port=`echo $topoint | awk '{print $2}'` echo "MyData $ip:$port $(cat /tmp/vpn2-$id-* | grep $myname | awk {'print $1'})" | nc $ip $port -u -p $1 -w 1 fi fi done
Runs automatically with a line in /etc/rc.local
nohup /--/connector2.sh 13013 > /var/log/connector2.log &
where 13013 is the port you want to listen to, /var/log/connector2.log is the log.
Script on nodes: # cat vpn5.sh
#!/bin/bash ######################## ### WARN='\033[37;1;41m' # END='\033[0m' # RED='\033[0;31m' # ${RED} # GREEN='\033[0;32m' # ${GREEN} # ################################################# ####################### ######################################################### al="echo readlink dirname ps grep awk kill md5sum shuf nc curl sleep openvpn tcpdump" ch=0 for i in $al; do which $i > /dev/null || echo -e "${WARN} $i ${END}"; which $i > /dev/null || ch=1; done if (( $ch > 0 )); then echo -e "${WARN}, ${END}"; exit; fi ####################################################################################################################### if [[ $1 == '' ]]; then echo -e "${WARN} ( , !) ${END} \t ${GREEN} /etc/rc.local nohup /< >/vpn5.sh > /var/log/vpn5.log 2>/dev/hull & ${END}"; exit; fi ABSOLUTE_FILENAME=`readlink -f "$0"` # DIR=`dirname "$ABSOLUTE_FILENAME"` # ################################################################# if [ ! -f "$DIR/secret.key" ]; then echo -e "${WARN} VPN- , : \ openvpn --genkey --secret secret.key : \ !!!${END} # ls -l secret.key -rw------- 1 root root 637 27 11:12 secret.key # chmod 600 secret.key"; exit; fi localport=`shuf -i 20000-65000 -n 1` # until [[ $count > 100 ]]; do ######################################################################################### name=`uname -n | md5sum | awk '{print $1}'` # ipsrv="45.141.103.45" # IP- portsrv="13013" # iftosrv=`ip route get $ipsrv | head -n 1 | sed 's|.*dev ||' | awk '{print $1}'` # netcattosrv="nc -u $ipsrv $portsrv -p $localport -w 1" # NC tcpdumptosrv="tcpdump -i $iftosrv udp and port $portsrv and src $ipsrv -n -c1 -A" # id=`echo $1| md5sum | awk '{print $1}'` # md5 ##################### ################################### echo -e "$(date) ${GREEN} 1 - ${END}" echo -e "$(date) ${GREEN} $iftosrv $localport ${END}" until [[ -n "$ip" && -n "$port" ]]; do if [[ -z "$data" ]]; then sleep 1 && echo "Ident: $id $name" | $netcattosrv > /dev/null & sleep 4 && pid=`ps xa | grep "$tcpdumptosrv" | grep -v grep | awk '{print $1}'` && if [[ -n $pid ]]; then kill $pid; echo "$(date) , ..."; fi & data=`$tcpdumptosrv` sleep 2 fi if [[ -n "$data" ]]; then #echo ": $data" data=`echo "$data" | grep "MyData" | sed 's|.*MyData ||' | sed 's/:/ /g'` myip=`echo "$data" | awk '{print $1}'` myport=`echo "$data" | awk '{print $2}'` ip=`echo "$data" | awk '{print $3}'` port=`echo "$data" | awk '{print $4}'` data='' if [[ -z $tst ]]; then echo -e "$(date) ${GREEN} IP: $myip : $myport ${END}"; fi if [[ -n $localport && -n $myport && -z $tst ]]; then if [[ $localport == $myport ]]; then echo -e "$(date) ${GREEN} $localport $myport , CONE NAT ${END}"; tst=1 else echo -e "$(date) ${RED} $localport $myport , SYMMETRIC NAT ${END}"; tst=1 fi fi fi if [[ -n "$ip" && -n "$port" ]]; then echo -e "$(date) ${GREEN} : IP: $ip : $port ${END}"; else sleep=`shuf -i 30-45 -n 1`; echo -e "$(date) ${RED} , $sleep ... ${END}"; sleep $sleep && pid=`ps xa | grep "$tcpdumptosrv" | grep -v grep | awk '{print $1}'` && if [[ -n $pid ]]; then kill $pid; fi & data=`$tcpdumptosrv` fi done ###################### iftopeer=`ip route get "$ip" | head -n 1 | sed 's|.*dev ||' | awk '{print $1}'` echo -e "$(date) ${GREEN} 2 - ${END}" echo -e "$(date) ${GREEN} : $myip:$myport -> $ip:$port ${END}" m=0 t=0 connect=0 until (( $connect >= 10 )); do if [ ! -f "$DIR/.mes" ]; then mes=`shuf -i 0-254 -n 1` echo $mes > $DIR/.mes else mes=`cat $DIR/.mes` fi netcattopeer="nc -u $ip $port -p $localport -w 1" tcpdumptopeer="tcpdump -i $iftopeer udp and port $localport and src "$ip" -n -c1 -A" until (( $m >= 10 )); do for w in {1..3}; do sleep 1 && echo "445Hi: $m $mes" | $netcattopeer > /dev/null; done & sleep 4 && pid=`ps xa | grep "$tcpdumptopeer" | grep -v grep | awk '{print $1}'` && sleep 10 && if [[ -n $pid ]]; then kill $pid && echo -e "$(date) ${RED} , ... ${END}"; fi & peer=`$tcpdumptopeer` if [[ -n $peer ]]; then newport=`echo "$peer" | grep " IP " | sed "s|.* $ip.||" | awk '{print $1}'` if (( $newport != $port )); then echo -e "$(date) ${WARN} $port -> $newport ${END}"; port=$newport; netcattopeer="nc -u $ip $port -p $localport -w 1" fi mm=`echo "$peer" | grep "445Hi: " | sed 's|.*445Hi: ||' | awk '{print $1}'` echo -e "$(date) ${GREEN} : $mm - : $mm+1 ${END}" if (( $m <= $mm )); then m=$(( $mm + 1 )); fi text=`echo "$peer" | grep "445Hi: " | sed 's|.*445Hi: ||' | awk '{print $2}'` else (( t++ )) echo -e "$(date) ${RED} $t ${END}" if (( $t > 5 ));then echo -e "$(date) ${RED} , ${END}"; m=10; connect=1012; ip=''; port=''; fi fi done if (( $connect < 1012 )); then echo -e "$(date) ${GREEN}: $m ${END}" if [[ $port > $myport ]]; then ipaddress="10.$text.$mes.1"; else ipaddress="10.$mes.$text.2"; fi m=12 for w in {1..9}; do echo "445Hi: $m $mes" | $netcattopeer > /dev/null; done killall nc echo -e "$(date) ${GREEN} $ipaddress...${END}" $(which lsmod) | grep tun || $(which modprobe) tun openvpn --remote $ip --rport $port --lport $localport \ --proto udp --dev tap --float --auth-nocache --verb 3 --mute 20 \ --ifconfig "$ipaddress" 255.255.255.252 \ --secret "$DIR/secret.key" \ --auth SHA256 --cipher AES-256-CBC \ --comp-lzo --ncp-disable --ping 10 --ping-exit 50 echo -e "$(date) ${RED} ... ${END}" m=0 t=0 (( connect++ )); fi done done
Runs automatically with a line in /etc/rc.local
sleep 15 && nohup /--/vpn5.sh <ID-> >/var/log/vpn5.log 2>/dev/hull &
where connection ID is any unique phrase for your connection, /var/log/vpn5.log is the connection log, ipsrv = "45.141.103.45" is the IP address of the node where connector2.sh is running (first script), sleep 15 - it is necessary for the network interfaces to rise during this time.
If everything is configured correctly, it will work like a clock: turned on and after a couple of minutes you have a connection with a remote host. The scripts use endless cycles and time delays, that is, the script works while the node is turned on and will try to restore it if the connection is lost. The scripts are not perfect, but the fact that this technology works gives me a lot of new ideas, for example:
- use the node to store critical data, and have access to it, for example, if the laptop is lost, all the data will be on the node
- use a remote computer desktop on a laptop
- organize connections with a friend’s computer and play a network game
- have access to a home security camera from a work computer or view a laptop’s webcam
Advantages:
- ease of setup and use
- traffic goes directly
- traffic is encrypted and protected from interception
- no need to forward ports
- the script starts automatically
Disadvantages:
- the main drawback is the use of VPS as a data source for the connection, but I think about it
- some providers block traffic between other providers (seen by mobile operators)
- when using one provider on both sides, it may not work, it depends on the type of NAT
- when using provider "hard" NAT may not work
There are plans to further develop the script to some system, minimize the waiting time for answers, tighten the encryption of the transmitted data, adapt it to Windows and Android, teach how to work through Proxy and the like.
I think against the background of a shortage of white IPv4 addresses, my decision will be relevant.