High-level overview of binary
27/2/2018
For an upcoming school project I am producing multiple small pdfs explaining (in extremely high-level) binary, hexadecimal, ASCII and possibly some on general assembler languages. I don't have the best grasp on these subjects, so producing them on their own is good practice for me. The first one is on binary.
These documents are produced using pandoc+beamer.
The first pdf can be found here.
Automated video-grepping for anime
21/2/2018
I've created a bash script that extracts the subtitles from an mkv video file (main use for me is anime), greps them for a specified word, parses them and extracts only that time frame from the source video file. The script outputs the clips created to out/clips. Around a week ago now I decided to finally go completely mouse-less - unplugged and packed away in its original box. I have been wanting to make this change for quite a while now and have slowly been transitioning things that I rely on a GUI for to more keyboard oriented approaches (or in some cases plugins that assist). The largest most substantial change I have made is my move back to qutebrowser. This sadly retires surf and tabbed (both fantastic suckless tools) from my suite of used software. Another major pain point is Reddit and Waterfox. I rely on various Firefox extensions for what I do on Reddit and changing to a browser that supports none of them is not viable, so instead I began using the VimFX plugin. This brings vim-like keyboard control to Waterfox along with hinting (allowing you to click on things with your keyboard). VimFX by default however does not support the new ModMail system, so I am going to have to investigate implementing hinting for that myself (I did roughly the same modifications on qutebrowser). I have began using Alpine as an email client (moving email entirely to the terminal), which will eventually retire Thunderbird. Currently, for replying to emails I still rely on Thunderbird (its suite of keyboard shortcuts makes this possible without a mouse). During the month of January 2018 I enrolled for a certificate in the South Australian branch of TAFE. Later in the month I was accepted a position and swiftly pointed to a website to create an account and fill in various pieces of private and confidential information. The website was built on the Salesforce platform. I've been using st as my terminal for a number of months now utilising the externalpipe patch along with the recommended open url handler. However, this url handler has always been inconsistent with selecting the appropriate application to spawn, so I created my own. It's extremely simple by design, but so far works great. To dip my feet back into C++/Qt I've created a simple RSS client for the HorribleSubs fansub group. The client parses their latest 720p (configurable) RSS feed for anime magnet download links and places the downloads all into a list. Using the checkboxes provided you select the torrents you want and press 'get'. The application then spawns your torrent client (in this case transmission) feeding it the magnet links. Task 4 is a subroutine implementation of multiplication. In addition of using a subroutine the program uses a list which is a defined portion of memory you can address. The subrotuine requires three bytes of memory - a byte for the multiplicand, multiplier and a storage byte for the result. The comments as always are very helpful if you're trying to understand it.
Excercise 3 is a simple yet important one. Jumping around code like a 'goto' statement in C isn't always the best way to go about doing things. The difference between jumping and calling is when you jump, the program never returns to the instruction after the jump, instead it flows on from that memory location. With a call to a subroutine you are able to 'RET' (return) either on condition or unconditionally. This is very similar to a function call in C. This program is a simple counter that utlises a subroutine call.
For excercise 2 and a few in the future I wanted to focus on implementing basic math functions. This task focuses on multiplication. Multiplication is nothing more than repeated addition, so, when broken down it's really quite simple. This implementation focuses on a two number operation, you need a multiplicand and a multiplier. In my program N1 stores the multiplican, N2 the multiplier. The steps to achieve N1*N2 are simple, add N1 to itself N2 times. I do not determine which number is bigger and reduce the number of iterations, although this may be a task for another day.
The main motivation behind this script is to automate the "every time x is said in anime y" YouTube videos. This doesn't extract only the single word though, instead the entire duration that line is played. It can help a lot however if you want to cut them down to only the word yourself.
How to use it:
-f/--file: the file you wish to extract clips from
-d/--directory: the directory you wish to extract clips from (does each mkv file in the directory)
-t/--track: the track number of the subtitles, you can find it with mkvinfo
-w/--word: the word or phrase you are grepping for (use quotes for phrases with spaces)
-m/--merge: merge all files in out/clips into out/out.mkv (not yet implemented)
Example of the script in use
You can find the script here.
No longer using a mouse
17/2/2018
I haven't particularly had the need for a mouse throughout this week - the one remaining pain point however is taking screen shots of regions of my screen. For this, although rare, currently I use my USB pen/tablet combination. My desk is much neater without a mouse and accompanying mouse pad.
qutebrowser hints for Reddit expand buttons
13/2/2018
Reddit expand buttons are not by default hinted. Customisable hinting for custom elements is not yet implemented (see #2773). So as a temporary work around I have replaced 'SELECTORS{...}' in qutebrowser/browser/webelem.py to:
SELECTORS = {
Group.all: ( 'a, area, textarea, select, input:not([type=hidden]), button, '
'frame, iframe, link, [onclick], [onmousedown], [role=link], '
'[role=option], [role=button], img, '
# Angular 1 selectors
'[ng-click], [ngClick], [data-ng-click], [x-ng-click],'
# Reddit expando selectors
'div[class="expando-button collapsed hide-when-pinned selftext"], div[class="expando-button hide-when-pinned selftext expanded"],'
'div[class="expando-button collapsed hide-when-pinned video"], div[class="expando-button hide-when-pinned video expanded"],'
'div[class="expando-button collapsed hide-when-pinned crosspost"], div[class="expando-button hide-when-pinned crosspost expanded"]'),
Group.links: 'a[href], area[href], link[href], [role=link][href]',
Group.images: 'img',
Group.url: '[src], [href]',
Group.inputs: ( 'input[type=text], input[type=email], input[type=url], '
'input[type=tel], input[type=number], '
'input[type=password], input[type=search], '
'input:not([type]), textarea'),
}
Allowing me to use hinting on Reddit expand elements.
Obtaining confidential information on other students in TAFE SA's Force system
9/2/2018
The site itself was interesting. On the surface it appears to be a forum platform including post/comment counts, followers, a complete profile to edit (including avatar) and other forum-esque features, however at least to the unprivileged user, completely locked down. We were informed that this website is how we will retrieve, submit and keep track of our work, reminiscent of Moodle. Currently I am able to upload files and manage them, however nothing else is public facing. Here is how the site looks to me currently (sitting on my profile page):
After playing around with the site for a little while as I do with any new service I use I stumbled upon a 404 page that is entirely not themed after the rest of the Force website:
This page appears to be a default Salesforce page. On the left side, a couple of interesting links appear. One of these links goes to one of the TAFE SA Force owners profile page. Nothing considerably interesting is found there, however the other link of interest proved to be something more.
The link pointed to a page located at "/community/s/detail/00X0X0000XXXXXX", This page held every piece of confidential information I provided when first signing up. This data included my full name, home address, mobile/home phone number, email address, employment status, education history, VET loan status and other confidential pieces of information. Disturbingly, this page also contains a widget and link that points Google maps to my house. Here is an image of the page and confidential information it contains (open the image to view it larger):
Interestingly, I have the ability to email myself, edit my information and clone the information(?). Every piece of information on the page I could edit. Cool, but mostly pointless, except that clone feature, what does that do? It seems like it allows me to clone my information into a separate entry of which I had complete ownership over. This could be an issue if spammed, but nothing worth complaining about.
While playing around with this page I decided to increment the last alphanumeric character in the URL, and to my surprise a profile appeared. Except this profile wasn't my own and was of another student. All of the information that was available on my page was available on this page as well. Their phone numbers, email address, home address, employment status, all there visible for me - I even had the permission to edit and clone this account. Now, this is problematic and worth complaining about on its own. But it didn't stop there, further incrementing of the last URL character took me to other profiles, going back a character and altering that let me access other profiles. The results were not always consistent, and I didn't spend that much time playing around, however the fact I had access to view and alter this information was appalling. Just to mention it again, these pages had Google map links to the students houses.
I swiftly reported this issue and it was fixed the following Monday (This was reported on a Saturday). While not a particularly entertaining or thrilling story, it goes to show just how poorly some companies and institutions handle information security. Something this simple may be easy to overlook and have incorrect default permissions in place, however the response I received hinted at them not knowing one was able to access their own information, let alone that of other students.
URL handle script that may be useful to some
1/2/2018
urlopen is available on git.
HorribleSubs RSS client for easy anime downloading
20/1/2018
The source code is available on git.
8085 assembly #5: Task #4
17/1/2018
; CALLING A ROUTINE AND PREPARE A MEMORY ADDRESS
; ROUTINE WILL REQUIRE 3 BYTES FIRST NUMBER -> SECOND NUMBER -> RESULT STORAGE
; ROUTINE WILL MULTIPLY M WITH M+1 AND STORE RESULT IN M+2
JMP START
; DEFINE A LIST TO STORE OUR REQUIRED BYTES
; DB = DEFINE BYTE
; DS = DEFINE STORAGE. OPERAND IS NUMBER OF BYTES
MULTI1: DB 05H ; MULTIPLICAND
DB 03H ; MULTIPLIER
DS 01H ; RESERVES "ONE BYTE OF STORAGE
START: LXI H,MULTI1 ; LOAD HL AS POINTER TO MULTI1 LIST MEMORY ADDRESS
MVI A,0 ; CLEAR A
MOV D,M ; STORE MULTIPLICAND IN D
INX H ; INCREASE MEMORY POINTER TO GET MULTIPLIER
MOV E,M ; STORE MULTIPLIER IN E
MVI C,00H ; COUNTER SET TO 0
CALL MULT ; CALL MULTIPLY FUNCTION
INX H ; INCREASE MEMORY POINTER TO RESULT STORAGE
MOV M,A ; STORE RESULT
HLT ; HALT
MULT: MOV B,A ; STORE A
MOV A,C ; MOVE COUNTER INTO A
CMP E ; COMPARE MULTIPLIER TO COUNTER
MOV A,B ; MOVE B STORAGE BACK INTO A
RZ ; RETURN IF ZERO FLAG SET (COUNTER == MULTIPLIER)
INR C ; INCREASE COUNTER
ADD D ; ADD MULTIPLICAND TO A
JMP MULT ; JUMP TO MULT
8085 assembly #4: Task #3
9/1/2018
; CALL ACTS LIKE A FUNCTION CALL IN C, A SUBROUTINE
; CALL COUNT AND COUNT TO C FROM B
JMP START
START: MVI B,00H ; COUNT FROM 0
MVI C,05H ; COUNT TO 5
MOV A,B ; COUNT FROM B
CALL CNT ; CALL COUNT SUBROUTINE
HLT ; HALT
CNT: CMP C ; COMPARE C TO A
RZ ; RETURN IF C==A
INR A ; A++
JMP CNT ; LOOP
8085 assembly #3: Task #2
8/1/2018
I produced two versions of todays task. The first is longer and over complicated, the second more simple and shorter. After I completed version #1 I realised a few things I could change and came up with version #2.
In particular I realised using a branching instructor is not the only way to determine if a number is zero, the status flags are in fact set when I do my DCR (decrement) operation. So the second interation does that, which saves a few lines of instructions (storing A, putting 00H in its place, comparing it to C and loading the original A back).
Iteration #1
; MULTIPLY TWO NUMBERS TOGETHER, STORE IN MEMORY AT 00H
; MULTIPLICATION IS NOTHING MORE THAN REPEATED ADDITION
JMP START
N1: EQU 05H ; MULTIPLICAND
N2: EQU 03H ; MULTIPLIER
N3: EQU 00H ; MEMORY LOCATION TO STORE RESULT
START: LXI H,N3 ; H AS MEMORY POINTER TO OUTPUT LOCATION 00H
MVI C,0 ; CLEAR C
MOV B,C ; CLEAR B
MOV A,B ; CLEAR A
MVI C,N2 ; MOVE N2 INTO C, THIS IS OUR COUNTER
ADD: MOV B,A ; STORE A IN B
MVI A,00H ; STORE 00H IN A
CMP C ; COMPARE C TO A
MOV A,B ; PUT B BACK INTO A
JZ END ; IF C == 0 JUMP TO END
ADI N1 ; ADD N1 TO A
DCR C ; DECREMENT C
JMP ADD ; JUMP TO ADD
END: STA N3 ; STORE A
HLT ; HALT
Iteration #2 (the better way)
; MULTIPLY TWO NUMBERS TOGETHER, STORE IN MEMORY AT 00H
; MULTIPLICATION IS NOTHING MORE THAN REPEATED ADDITION
JMP START
N1: EQU 05H ; MULTIPLICAND
N2: EQU 03H ; MULTIPLIER
N3: EQU 00H ; MEMORY LOCATION TO STORE RESULT
START: LXI H,N3 ; H AS MEMORY POINTER TO OUTPUT LOCATION 00H
MVI C,0 ; CLEAR C
MOV A,C ; CLEAR A
MVI C,N2 ; MOVE N2 INTO C, THIS IS OUR COUNTER
ADD: ADI N1 ; ADD N1 TO A
DCR C ; DECREMENT C
JNZ ADD ; IF C == 0 JUMP TO END
STA N3 ; STORE A IN MEMORY AT N3
HLT ; HALT
