Sorting images from log files

24/4/2017

Recently i've been wanting to collect and store images (mostly reaction images/anime screenshots) that have been posted in a specific IRC channel.
To accomplish this I decided to take the DIY route and create my own scripts/applications.

Collecting the images

The first issue I encountered was collecting the images from IRC logs. Being a Linux user I knew this wouldn't be too difficult, and it wasn't, infact it was amazingly simple. First thing to do was collect all the log files from ZNC and place them into a directory. This was done with the following command
cp ~/.znc/moddata/logs/zncname_servername_#channel* ~/whereever/you/want
With the logs collected the next step was parsing the urls of jpgs, gifs and pngs out of the logs. From there, a directory for the images we download has to be made. The last step is downloading the images, this was accomplished using wget with -i indicating a file of urls to download and --directory-prefix indicating the location images should be downloaded to.
This was done through the following bash script.
#!/bin/bash
echo 'run this in the directory containing all your logs (not recursive)'
cat *.log | egrep -o 'https?://[^ ]+.png' >> files.txt
cat *.log | egrep -o 'https?://[^ ]+.jpg' >> files.txt
cat *.log | egrep -o 'https?://[^ ]+.gif' >> files.txt
mkdir images
wget -i "files.txt" --directory-prefix=images


Sorting the images

With the thousands upon thousands of images collected (at this point I realised this project was unrealistic, I only continued because the application and scripts created are actually useful) I needed a way to name them. I like to keep my image names descriptive, perfect for finding them in a short period of time (aka sending them to people in response to questions). Using Qt and C++ I created a simple image viewer that supports GIF/PNG/JPG that allows me to select a directory and rename each image. Renamed images are moved into a directory named "sorted". You can also delete images, this was a crucial feature as a number of the images downloaded from the logs were not of any use to my collection.


The code for this project can be found on GitHub