Backup procedure
14/6/2018
A year or two ago I my bulk data storage drive died - it was 2TB and held information spanning back to around 2013 when I first got this computer. Of course, none of the data was backed up. I thought this would hurt, but it didn't. I've not thought twice about the data I had lost. I guess none of it was important.
Today, things are a little different. I keep semi up-to-date copies of things that are important to me (namely my anime collection and /home) on an external 1TB drive that I always have on my body. My thought process here is if both the drive I carry and my computer are destroyed at the same time, I'd probably be dead too.
I also keep a similarly semi up-to-date backup of my server on the same drive. This backup is never as up-to-date as I'd like.
I use cronjobs and bash scripts to perform daily backups of my personal computer and server. These are stored on a drive dedicated to backups always mounted on my computer, in the case of my server - a mere directory. Every now and then I copy these backups to the drive I carry everywhere. These are single tar files gzip compressed. I completely understand having the backups mounted and accessible is not the right way to do things.
I keep these backups for a short period after which they are deleted and replaced with newer ones.
Local personal computer backup cronjob and script:
# cronjob # runs everyday at 1pm 0 13 * * * /home/daniel_j/programming/bash/backup/backup.sh # delete backups older than 5 days # runs every day at 3pm 0 15 * * * find /mnt/backups/tar_backups/old_backups/ -type f -mtime +5 -delete # backup script #!/bin/bash # move the last backup performed into the old backups directory mv /mnt/backups/tar_backups/*.tar.gz /mnt/backups/tar_backups/old_backups/ #backups tar -cvpzf /mnt/backups/tar_backups/home-backup-$( date '+%Y-%m-%d_%H-%M-%S' ).tar.gz /home > /dev/null echo "buzz=500" >/dev/tcp/localhost/3001(As a side note, the "echo "buzz" > /dev/tcp/localhost/3001" sends a command to an arduino that sounds a buzzer. I use it as an alert. I'll write more about this in another post.)
Server:
# cronjob # backup @daily /home/username/scripts/backup.sh # delete backups older than 2 days @daily find /home/username/backups/old_backups/ -type f -mtime +2 -delete # backup script #!/bin/bash # move the last backups performed into the old backups directory # (I backup both directories and an sql dump) mv /home/username/backups/*.tar.gz /home/username/backups/old_backups/ mv /home/username/backups/*.sql /home/username/backups/old_backups/ # backups # dump sql databases /usr/bin/mysqldump --all-databases > /home/username/backups/dump-$( date '+%Y-%m-%d_%H-%M-%S' ).sql -u root -pr00tpassw0rd # I backup everything valuable on / tar -cvpzf /home/username/backups/backup-$( date '+%Y-%m-%d_%H-%M-%S' ).tar.gz --exclude=/home/username/backups --exclude=/proc --exclude=sys --exclude=/mnt --exclude=/media --exclude=/run --exclude=/dev --exclude=/var/www/desu/f --exclude=/home/username/old_server --one-file-system / > /dev/nullAs for my anime collection, that is simply an rsync command.
Anime tracker
8/6/2018
MAL (MyAnimeList) recently forced every user with an account to reset their password, they say this is out of caution regarding an exploit they found within their API. A few days later the entire website disappears for days, completely inaccessible. They gave no explanation for this and weeks later many of their services are still unavailable.
Clearly, this is beyond acceptable. Through this disaster however came a new project idea.
I've began work on creating my own anime static anime tracker. You can find the goals of the project here.
There is a live version of the project here.
The source code can be found here, licensed under GPL2.
Miscellaneous RGB controller additions
13/4/2018
A month or two back I obtained a monitor arm in order to further reduce the clutter on my desk. Once setup I decided to get my RGB controller project back in a working order.
I ended up putting the strip to the underside of my monitor as well as mounting the arduino and accompanying breadboards on the back.
I added an oled display on the bottom of my monitor which currently displays the time and current song time/duration. This is achieved with a simple bash script. The arduino was programmed accordingly to receive and display the data. I also added a switch that controls the power going to the LED's. There is a slight leak of power when the RGB channels are set to 0 (meaning off), this switch allows me to kill the power supply entirely. Finally, I added a female power jack to make supplying the required 12v easier.
The code as always is available on git, now licensed under GPL 3.
The wiring is an absolute hackjob. I don't particularly enjoy wiring or working with components, let alone a soldering iron.
More things to mention
9/4/2018
New domain
Recently I acquired the domain 'gnupluslinux.com'. On a whim I had checked if such a domain was registered, .org was, however .com was available. I really had to have it.Currently I have https://i.use.gnupluslinux.com point to a listing of the software I use and the root domain https://gnupluslinux.com presents an explanation of what GNU plus Linux is. I also host my screenshots on the img subdomain currently.
The source of the website and its subdomains is appropriately available under the GNU GPL version 2 on git. If you're interested in hosting something on the domain (or optional subdomain), feel free to send a PR on GitHub or via email.
snake in c
I've been wanting some terminal games recently, instead of obtaining some from online it's much more entertaining to make my own. The first was snake.You can see an example of the game here and obtain the source code on git.
General things to mention
19/3/2018
No mouse
It has been well over a month now since I packed my mouse away and relied entirely on my keyboard for everything. After a week or so my teething problems were mostly sorted out, however to this day I still find myself using my tablet for screenshots and Reddit ModMail. However I rarely perform these two tasks and they are really non-issues.I have bound mod(+shift)+volume up/down (my keyboard has a volume wheel) to control the cursor for the other rare times in which I need to click on something (button on a website that can't be hinted for example) and so far that has worked well. I'm really quite happy with my progress with learning and relying on new keyboard oriented tools and shortcuts for doing everyday tasks. I have no desire or intention to go back to using a mouse.
Operating system
Two or so weeks ago I migrated my Arch Linux system to Parabola Linux-Libre. Parabola Linux-Libre is based on Arch Linux and is on the Free Software Foundations list of approved GNU/Linux distributions. The main modification is the use of the Linux-Libre kernel. The stock Arch kernel has proprietary blobs shipped with it, the Libre kernel strips those out. Parabola also comes with a package named your-freedom. your-freedom conflicts with every package available in the Arch repository that isn't free software, meaning you cannot install non-free software without removing your-freedom. This name is absolute genius. The switch was mostly painless - the hardest part was going through everything in the AUR I have installed and removing things that aren't free. However, now I am running (as far as I can tell) completely free software! Of course, apart from my bios... and I still have Intel ME... I'll get a t60 or x200 and libreboot it one day.School network
Similar to the exploit in TAFE's Force system I recently found, my school had a directory of private and confidential files free for anyone with an unprivileged network account to access. The network share held each staff members personal storage directory. These directories contained countless pieces of private and confidential documents on both students and the staff members themselves. This has been an existing issue since at least 2016. I reported the issue and it was fixed the next time I was in school.Directory permissions. They aren't rocket science. I fear what else lurks open for others to view on the network.
It's nice to get things off my chest.
High-level overview of binary
27/2/2018
For an upcoming school project I am producing multiple small pdfs explaining (in extremely high-level) binary, hexadecimal, ASCII and possibly some on general assembler languages. I don't have the best grasp on these subjects, so producing them on their own is good practice for me. The first one is on binary.
These documents are produced using pandoc+beamer.
The first pdf can be found here.
Automated video-grepping for anime
21/2/2018
I've created a bash script that extracts the subtitles from an mkv video file (main use for me is anime), greps them for a specified word, parses them and extracts only that time frame from the source video file. The script outputs the clips created to out/clips. Around a week ago now I decided to finally go completely mouse-less - unplugged and packed away in its original box. I have been wanting to make this change for quite a while now and have slowly been transitioning things that I rely on a GUI for to more keyboard oriented approaches (or in some cases plugins that assist). The largest most substantial change I have made is my move back to qutebrowser. This sadly retires surf and tabbed (both fantastic suckless tools) from my suite of used software. Another major pain point is Reddit and Waterfox. I rely on various Firefox extensions for what I do on Reddit and changing to a browser that supports none of them is not viable, so instead I began using the VimFX plugin. This brings vim-like keyboard control to Waterfox along with hinting (allowing you to click on things with your keyboard). VimFX by default however does not support the new ModMail system, so I am going to have to investigate implementing hinting for that myself (I did roughly the same modifications on qutebrowser). I have began using Alpine as an email client (moving email entirely to the terminal), which will eventually retire Thunderbird. Currently, for replying to emails I still rely on Thunderbird (its suite of keyboard shortcuts makes this possible without a mouse). During the month of January 2018 I enrolled for a certificate in the South Australian branch of TAFE. Later in the month I was accepted a position and swiftly pointed to a website to create an account and fill in various pieces of private and confidential information. The website was built on the Salesforce platform.
The main motivation behind this script is to automate the "every time x is said in anime y" YouTube videos. This doesn't extract only the single word though, instead the entire duration that line is played. It can help a lot however if you want to cut them down to only the word yourself.
How to use it:
-f/--file: the file you wish to extract clips from
-d/--directory: the directory you wish to extract clips from (does each mkv file in the directory)
-t/--track: the track number of the subtitles, you can find it with mkvinfo
-w/--word: the word or phrase you are grepping for (use quotes for phrases with spaces)
-m/--merge: merge all files in out/clips into out/out.mkv (not yet implemented)
Example of the script in use
You can find the script here.
No longer using a mouse
17/2/2018
I haven't particularly had the need for a mouse throughout this week - the one remaining pain point however is taking screen shots of regions of my screen. For this, although rare, currently I use my USB pen/tablet combination. My desk is much neater without a mouse and accompanying mouse pad.
qutebrowser hints for Reddit expand buttons
13/2/2018
Reddit expand buttons are not by default hinted. Customisable hinting for custom elements is not yet implemented (see #2773). So as a temporary work around I have replaced 'SELECTORS{...}' in qutebrowser/browser/webelem.py to:
SELECTORS = {
Group.all: ( 'a, area, textarea, select, input:not([type=hidden]), button, '
'frame, iframe, link, [onclick], [onmousedown], [role=link], '
'[role=option], [role=button], img, '
# Angular 1 selectors
'[ng-click], [ngClick], [data-ng-click], [x-ng-click],'
# Reddit expando selectors
'div[class="expando-button collapsed hide-when-pinned selftext"], div[class="expando-button hide-when-pinned selftext expanded"],'
'div[class="expando-button collapsed hide-when-pinned video"], div[class="expando-button hide-when-pinned video expanded"],'
'div[class="expando-button collapsed hide-when-pinned crosspost"], div[class="expando-button hide-when-pinned crosspost expanded"]'),
Group.links: 'a[href], area[href], link[href], [role=link][href]',
Group.images: 'img',
Group.url: '[src], [href]',
Group.inputs: ( 'input[type=text], input[type=email], input[type=url], '
'input[type=tel], input[type=number], '
'input[type=password], input[type=search], '
'input:not([type]), textarea'),
}
Allowing me to use hinting on Reddit expand elements.
Obtaining confidential information on other students in TAFE SA's Force system
9/2/2018
The site itself was interesting. On the surface it appears to be a forum platform including post/comment counts, followers, a complete profile to edit (including avatar) and other forum-esque features, however at least to the unprivileged user, completely locked down. We were informed that this website is how we will retrieve, submit and keep track of our work, reminiscent of Moodle. Currently I am able to upload files and manage them, however nothing else is public facing. Here is how the site looks to me currently (sitting on my profile page):
After playing around with the site for a little while as I do with any new service I use I stumbled upon a 404 page that is entirely not themed after the rest of the Force website:
This page appears to be a default Salesforce page. On the left side, a couple of interesting links appear. One of these links goes to one of the TAFE SA Force owners profile page. Nothing considerably interesting is found there, however the other link of interest proved to be something more.
The link pointed to a page located at "/community/s/detail/00X0X0000XXXXXX", This page held every piece of confidential information I provided when first signing up. This data included my full name, home address, mobile/home phone number, email address, employment status, education history, VET loan status and other confidential pieces of information. Disturbingly, this page also contains a widget and link that points Google maps to my house. Here is an image of the page and confidential information it contains (open the image to view it larger):
Interestingly, I have the ability to email myself, edit my information and clone the information(?). Every piece of information on the page I could edit. Cool, but mostly pointless, except that clone feature, what does that do? It seems like it allows me to clone my information into a separate entry of which I had complete ownership over. This could be an issue if spammed, but nothing worth complaining about.
While playing around with this page I decided to increment the last alphanumeric character in the URL, and to my surprise a profile appeared. Except this profile wasn't my own and was of another student. All of the information that was available on my page was available on this page as well. Their phone numbers, email address, home address, employment status, all there visible for me - I even had the permission to edit and clone this account. Now, this is problematic and worth complaining about on its own. But it didn't stop there, further incrementing of the last URL character took me to other profiles, going back a character and altering that let me access other profiles. The results were not always consistent, and I didn't spend that much time playing around, however the fact I had access to view and alter this information was appalling. Just to mention it again, these pages had Google map links to the students houses.
I swiftly reported this issue and it was fixed the following Monday (This was reported on a Saturday). While not a particularly entertaining or thrilling story, it goes to show just how poorly some companies and institutions handle information security. Something this simple may be easy to overlook and have incorrect default permissions in place, however the response I received hinted at them not knowing one was able to access their own information, let alone that of other students.
