Sort photos by data from EXIF + PHP
I want to share my experience of sorting photos using a script in PHP
There comes a moment when there are not so many photographs, but catastrophically many.
One day I decided to sort my entire archive of digital photos, accumulated over 20 years, and realized that in all that time I had accumulated 112,000 photos at 435 gigabytes.
Moreover, some of them are more or less sorted, for example, photos from an SLR camera, by folders with names and dates, while the other part of the photos that were imported from iphone / android are not named and sorted, often it’s just a giant folder 10 gigabytes, with a couple of thousand files inside, and it’s a pity to delete and sort it out.
I started looking for automatic sorting tools and realized that all good services, like Picasa, had already been bought and closed by Google, of course you can upload everything to them on Google. Photos, however, there are problems there, not all are searched and generally half of the functions are missing from to be in Picasa, and if you are still worried that your photos will be recognized and used, uploading to the web is not your way at all.
As a result, it was decided to write a small script that will sort everything, at first I thought about a shell script, but realizing that I would need it, EXIF decided to return to the good old PHP.
First, I went the simplest way, take all the files, see the creation date and scatter along the nested paths:
There were several problems:
It was decided to take the date from EXIF, rename and touch for files to set the date from exif, and also check files for duplicates using md5.
In principle, PHP already has an exif extension in the library set, so nothing supernatural was foreseen
Everything would be fine, 500 gigabytes of photos were sorted and cleaned of duplicates in a couple of hours, but then I remembered the old folders that contained the name of the region where the photo shoot was held, and thought, why not get the names of cities from geodata?
The coordinates are easy to find in the files, they are in Exif in GPSLongitude and GPSLatitude, but we must not forget that they are stored there in degrees, minutes and seconds, so you need to use the functions to convert coordinates to decimal.
The second question, what to do with the coordinates, how to get the name of the city?
Geocoder from Yandex comes to the rescue, but be careful with the limits and terms of use.
In order not to kill Yandex with millions of queries, we cache the data in MySql, rounding the coordinates to 3 decimal places, that is, 43.161 - 19.182 is enough to determine the city, and thus for 110,000 photos I got only 1,500 geometries.
The appearance of the folders is something like this:
In fact, if you take on this product, you can do it for months, it took me a day to write a script and optimize the storage of photos.
From the plans: adding geotags to existing photos, re-sorting the current archive of photos, finding duplicates among clipped images.
All project files are available on GitHub
Do not hit me hard, this is my first completely open source project, if something is posted or written incorrectly, tell me, and now everything is imprisoned for the Windows runtime environment with encoding 1251.
There comes a moment when there are not so many photographs, but catastrophically many.
Background
One day I decided to sort my entire archive of digital photos, accumulated over 20 years, and realized that in all that time I had accumulated 112,000 photos at 435 gigabytes.
Moreover, some of them are more or less sorted, for example, photos from an SLR camera, by folders with names and dates, while the other part of the photos that were imported from iphone / android are not named and sorted, often it’s just a giant folder 10 gigabytes, with a couple of thousand files inside, and it’s a pity to delete and sort it out.
I started looking for automatic sorting tools and realized that all good services, like Picasa, had already been bought and closed by Google, of course you can upload everything to them on Google. Photos, however, there are problems there, not all are searched and generally half of the functions are missing from to be in Picasa, and if you are still worried that your photos will be recognized and used, uploading to the web is not your way at all.
As a result, it was decided to write a small script that will sort everything, at first I thought about a shell script, but realizing that I would need it, EXIF decided to return to the good old PHP.
Task №1 - Expand all files by dates
First, I went the simplest way, take all the files, see the creation date and scatter along the nested paths:
$file_list = $files->getDirContents($config['photos.unsorted']); foreach ($file_list as $key => $value) { moveImageFile($value); } function moveImageFile($filename) { $dt= new DateTime(); $dt->setTimestamp(filectime($filename)); $start_path = $this->config['photos']; $year = $start_path."\Year".$dt->format('Y'); if (!is_dir($year)) mkdir($year); $month = $year."\\".$dt->format('Ym-F'); if (!is_dir($month)) mkdir($month); $path = $month."\\".$dt->format('Ym-d'); if (!is_dir($path)) mkdir($path); } $full_path = getUniqueFilename($filename, $path, $dt, 0); copy($filename, $full_path);
There were several problems:
- Some files had an incorrect creation date
- If you do copy, a new file is created by the current date.
- Files can have duplicates with the same creation time
Problem number 2 - Get the date from Exif
It was decided to take the date from EXIF, rename and touch for files to set the date from exif, and also check files for duplicates using md5.
In principle, PHP already has an exif extension in the library set, so nothing supernatural was foreseen
$dt = DateTime::createFromFormat('Y:m:d H:i:s', $exif['DateTime']); $start_path = $this->config['photos.exif']; $is_exif = true; if (md5_file($filename) == md5_file($full_path)) return false; rename($filename, $full_path); touch($full_path, $dt->getTimestamp());
Everything would be fine, 500 gigabytes of photos were sorted and cleaned of duplicates in a couple of hours, but then I remembered the old folders that contained the name of the region where the photo shoot was held, and thought, why not get the names of cities from geodata?
Task №3 - Countries, cities and regions from EXIF geodata
The coordinates are easy to find in the files, they are in Exif in GPSLongitude and GPSLatitude, but we must not forget that they are stored there in degrees, minutes and seconds, so you need to use the functions to convert coordinates to decimal.
function getGps($exifCoord, $hemi) { $degrees = count($exifCoord) > 0 ? $this->gps2Num($exifCoord[0]) : 0; $minutes = count($exifCoord) > 1 ? $this->gps2Num($exifCoord[1]) : 0; $seconds = count($exifCoord) > 2 ? $this->gps2Num($exifCoord[2]) : 0; $flip = ($hemi == 'W' or $hemi == 'S') ? -1 : 1; return $flip * ($degrees + $minutes / 60 + $seconds / 3600); }
The second question, what to do with the coordinates, how to get the name of the city?
Geocoder from Yandex comes to the rescue, but be careful with the limits and terms of use.
$url = "https://geocode-maps.yandex.ru/1.x/"; $apikey = require('../config/apikey.php'); $json = array( 'geocode' => $lon.",".$lat, 'kind' => 'locality', 'apikey' => $apikey, 'results' =>'1', 'skip' => '0', 'format' => 'json' ); $response = file_get_contents($url."?".http_build_query($json));
In order not to kill Yandex with millions of queries, we cache the data in MySql, rounding the coordinates to 3 decimal places, that is, 43.161 - 19.182 is enough to determine the city, and thus for 110,000 photos I got only 1,500 geometries.
The appearance of the folders is something like this:
- D: \ photos \ photos_exif \ Year2019 \ 2019-09-September \ 2019-09-23-Bosnia and Herzegovina, Republika Srpska, Foca \
- D: \ photos \ photos_exif \ Year2019 \ 2019-08-August \ 2019-08-25-Albania, Durres region, Kruja \
- D: \ photos \ photos_exif \ Year2018 \ 2018-10-October \ 2018-10-06-Russia, Moscow Region, Balashikha \
Instead of a conclusion
In fact, if you take on this product, you can do it for months, it took me a day to write a script and optimize the storage of photos.
From the plans: adding geotags to existing photos, re-sorting the current archive of photos, finding duplicates among clipped images.
All project files are available on GitHub
Do not hit me hard, this is my first completely open source project, if something is posted or written incorrectly, tell me, and now everything is imprisoned for the Windows runtime environment with encoding 1251.
All Articles