simple csv parse
30/10/2019
I store purchases I make in a spreadsheet. The spreadsheet is becoming a pain to use so I am creating a program to store them in a database and prettyprint purchases.
The first piece needed was to parse the csv output of the spreadsheet, I did this with a simple C program:
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
struct purchase
{
char *product;
char *website;
char *date;
char *cost;
char *notes;
};
void
parsepurchase(struct purchase *p, char *line)
{
char *modline;
char *bup;
size_t linesize;
size_t len;
linesize = strlen(line);
modline = malloc(linesize + 1);
bup = modline;
strcpy(modline, line);
modline = strtok(modline, ",\"");
p->product = malloc(strlen(modline) + 1);
strcpy(p->product, modline);
modline = strtok(NULL, ",\"");
p->website = malloc(strlen(modline) + 1);
strcpy(p->website, modline);
modline = strtok(NULL, ",\"");
p->date = malloc(strlen(modline) + 1);
strcpy(p->date, modline);
modline = strtok(NULL, ",\"");
p->cost = malloc(strlen(modline) + 1);
strcpy(p->cost, modline);
modline = strtok(NULL, ",\"");
p->notes = malloc(strlen(modline) + 1);
strcpy(p->notes, modline);
/* remove unwanted newline when note exists */
len = strlen(p->notes) - 1;
if (p->notes[len] == '\n')
p->notes[len] = '\0';
printf("%s, %s, %s, %s, %s\n",
p->product,
p->website,
p->date,
p->cost,
p->notes);
free(bup);
}
void
freepurchases(struct purchase *purchases, size_t count)
{
size_t i = 0;
while (i != count)
{
free(purchases[i].product);
free(purchases[i].website);
free(purchases[i].date);
free(purchases[i].cost);
free(purchases[i].notes);
i++;
}
}
int
main(int argc, char **argv)
{
FILE *file;
struct purchase purchases[512]; /* max 512 entries for now */
char *line;
size_t len;
ssize_t read;
int i;
if (argc != 2)
{
printf("use: %s csvfile\n", argv[0]);
}
file = fopen(argv[1], "r");
if (!file)
{
fprintf(stderr, "unable to open csv file\n");
exit(EXIT_FAILURE);
}
line = NULL;
len = 0;
i = 0;
while ((read = getline(&line, &len, file)) != -1)
{
parsepurchase(&purchases[i], line);
i++;
}
freepurchases(purchases, i);
fclose(file);
free(line);
return EXIT_SUCCESS;
}
more updates
12/9/2019
UPS battery failing
My CyberPower BR650ELCD UPS battery finally had its battery die after many years, no idea how many, 4-5? I purchased a new battery for it.I also purchased a second UPS, the APC BX700U. It is generally the same specs, 700va 390W, though this one is not a user replaceable battery.
I've now put the CyberPower UPS upstairs for my camera server + router/modem and kept the APC in my room.
4-Bit Adder
A month or so ago I started building a 4-Bit Adder circuit on a breadboard using logic gates, I've only implemented 2 bits so far, education and work became busy. This was my first go at using solid-core wire, so it is pretty ugly.
Manga
I also have my first shelf filled! These are the series I hold dearest to my SOL loving heart.
I don't know if I will continue getting manga or go towards artwork for a while, my walls are bare. I'd like to get some figurines, but quality ain't cheap. I've found many wall scrolls I'd like to get, I should commit to it.
old project
An old project of mine animegrep (which I named 'everytime-desu' originally) has been picked up by someone else and improved upon a little. You can find their fork here.
some things
14/7/2019
A while ago I started writing a project that I intended to implement into sic that would warn me when I attempt to repost a link into an IRC channel. My interest in the project faded, so I likely will never complete it.
The source code can be found in my git repo.
A month or two ago I pulled the plug on self-hosting my email and now use another provider. It isn't ideal, but whatever.
I started a manga collection and have very quickly amassed 19 volumes of series I loved. I also picked up a few volumes from the Manga guide to series. I'm currently reading the volumes on electricity and microprocessors. Very fun and educational.
I also a while ago acquired a couple stm32 dev boards that I intended to learn and use, problem being shipping took a long time and my interest bled, I need to find motivation for that again.
I am also quickly nearing 100 posts here, currently this is 97. Now that means nothing as half of it is dribble, but my sitegenerator by memory will break when it has to handle the 100th post. I hope my memory is wrong, I really don't want to look at that disgusting code again.
xwake
17/4/2019
I created a simple program in C that prevents the X11 screensaver from starting between two times. In hindsight cronjobs would have been easier.
The source code can be found in my git repo.
Product Inventory remake
11/4/2019
At the end of 2018 I was asked to create an inventory system for someone that wanted to document what they owned. They couldn't find a system that had the features they wanted. So I did create it, it works fine and is still active and functional today. However, the project is written in PHP and is a mess. Since then I've learnt a lot more about realtional databases and how to properly design them, so I recreated the entire system in C++ and Qt.
Currently the filtering options aren't implemented, but the table generation, category loading and database management settings work.
The entire interface in the filter tab
The database tab
The source code can be found in my git repo.
X bell listener
26/3/2019
I wanted to be able to run a script when the X bell is rung, so I made a program that listens for the X bell notify event and runs a user defined program when it is caught.
The source code can be found in my git repo.
urlopener re-written in C
8/3/2019
For the longest time I was using a urlopener script I had written in Python. I pipe links from my terminal into dmenu, then into my urlopener. The urlopener launches certain programs depending on the domain and file extensions. I recently thought it'd be fun to rewrite it in C. It was fun.
The source code can be found in my git repo.
Terrible, hacky database syncing
30/1/2019
Someone in my family wanted a custom inventory system made that was accessible from any device and had some features they couldn't find in any pre-made solution so I made my own. It works great (code not distributed, if you want it ask, GPL3), however prior to creating it they never mentioned wanting to view it away from their LAN. So I had to come up with a solution.
The server that the website (it's PHP/mysql) is located on has no access to external servers, so I had to route everything through my own computer.
My solution was cronjobs, sqldumps and sftp. Perfectly hacky.
Every hour (*/59 * * * *, each computer has a different localtime, so there isn't any concerns there) the server hosting the website creates an sql dump of the database and backups the uploaded images, my local computer then pulls that dump and backup and uploads the sql dump to my external server, which then imports it.
The images are not uploaded to the external server, but this is fine, the user indicated they only need access to the textual data off-site. It would be just too much data otherwise.
I also created a cut-down viewer for the data removing the ability to view images, add, edit and delete items.
sql dump and images backup:
name=$(date +"%d-%b-%Y-%H-%M") mysqldump -u inventory -ppasswordhere inventory > /home/user/inventory.sql tar -zcvf /webuser/backups/inventory_directory.tar.gz /var/www/html/inventory cp /webuser/backups/inventory_directory.tar.gz /home/user/inventory_directory.tar.gzdownloading and uploading the backups and sql dump:
sftp user@tewi:/home/user <<EOF get /home/user/inventory.sql /home/daniel_j/programming/inventory.sql get /home/user/inventory_directory.tar.gz /home/daniel_j/programming/inventory_directory.tar.gz exit EOF sftp -oPort=port user@danieljon.es:/home/user <<EOF put /home/daniel_j/programming/inventory.sql exit EOFImporting the database:
mysql -u inventory -ppasswordhere inventory < /home/user/inventory.sql
listen.moe song history
10/1/2019
I had someone who listens to listen.moe complain that they had heard a song they liked but couldn't get the name of because they were driving. The website only gives you the last 3 songs played so I used their API to parse their song list and display every one that was played on a particular day in order. They found the song, and it sure is great.
getsongs.py
#!/usr/bin/python2.7
import json
import urllib2
import codecs
def request(url, header, jsonpayload):
baseurl = "https://listen.moe/api/{}";
req = urllib2.Request(baseurl.format(url));
req.add_header("Content-Type", "application/json");
req.add_header("Accept", "application/vnd.listen.v4+json");
if header is not None:
req.add_header(header[0], header[1]);
response = urllib2.urlopen(req, json.dumps(jsonpayload) if jsonpayload is not None else None);
return response;
loginpayload = {
"username": "name",
"password": "password"
}
def main():
# login and get response with our token
response = request("login", None, loginpayload);
# extract token
token = json.loads(response.read());
token = token["token"];
# request songs list
response = request("songs", ["Authorization", "Bearer " + token], None);
response = json.loads(response.read());
# dump song list to file
with codecs.open("songs.json", 'w', encoding="utf-8") as f:
json.dump(response, f, indent=2, sort_keys=True, ensure_ascii=False);
exit();
if __name__ == "__main__":
main();
parse.py
#!/usr/bin/python2.7
import json
date = "2019-01-09";
songs = [];
def main():
infile = "songs.json";
with open(infile) as f:
data = json.load(f)
for song in data["songs"]:
if date in song["lastPlayed"]:
songs.append(song);
# sort songs by timestamp
songs.sort(key=lambda x:x["lastPlayed"]);
for song in songs:
print("{} last played {}".format(song["title"].encode("utf-8"), song["lastPlayed"]));
if __name__ == "__main__":
main();
A simple binary counter and LED matrix
20/12/2018
Using this same technique I emulated an 8x8 LED matrix and created a bouncing ball demo. The code can be found in my git repo.
#include <stdio.h>
#include <unistd.h>
// website generator breaks with unicode, so fill in the unicode characters yourself
#define CIRCLE "CIRCLE"
#define BLACKCIRCLE "BLACKCIRCLE"
void
printleds(unsigned char value)
{
printf("\0337");
printf("%s", "\033[1;0F");
printf("%s", "\033[0J");
printf("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n",
(value & 1 << 7) ? BLACKCIRCLE : CIRCLE,
(value & 1 << 6) ? BLACKCIRCLE : CIRCLE,
(value & 1 << 5) ? BLACKCIRCLE : CIRCLE,
(value & 1 << 4) ? BLACKCIRCLE : CIRCLE,
(value & 1 << 3) ? BLACKCIRCLE : CIRCLE,
(value & 1 << 2) ? BLACKCIRCLE : CIRCLE,
(value & 1 << 1) ? BLACKCIRCLE : CIRCLE,
(value & 1 << 0) ? BLACKCIRCLE : CIRCLE
);
printf("\0338");
}
int
main(void)
{
unsigned char value = 0;
while (1)
{
printleds(value);
value++;
if (value >= 255)
value = 0;
sleep(1);
}
return 0;
}
