Generating a SSH key and uploading it to a server

Generate a key with:

ssh-keygen -t rsa

When you are entering the values for the key, remember to set the name and path appropriately.
Upload the public part to your server using:

ssh-copy-id -i ~/.ssh/root_key.pub root@1.2.3.4

Whatever you set the name and path to in the ssh-keygen part you will need to make sure it replaces the ~/.ssh/root_key bit in the second command.

GitLab Docker Container on Fedora 26 Server

I’ve set up a NUC in the house to replace a Digital Ocean droplet.
My first job was to set up a headless Fedora 26 server. Currently it’s running Nextcloud which I set up using instructions from: marksei.com, and a docker container with Gitlab on it, based on instrucitons from this site.

To get the docker container to run I sued the following code:

docker run --detach --hostname gitlab.my.domain --publish 1443:443 --publish 180:80 --publish 122:22     --name gitlab     --restart always     --volume /srv/gitlab/config:/etc/gitlab:Z     --volume /srv/gitlab/logs:/var/log/gitlab:Z     --volume /srv/gitlab/data:/var/opt/gitlab:Z    gitlab/gitlab-ce:latest

Notice that the ports are translated to:
1443 from 443 (https)
180 from 80 (http)
122 from 22 (ssh)

…because I was already running https, http and ssh on the server.
mark the repo as the origin by going into the directory and using the following command for SSH

git remote add origin ssh://git@yourIPADDRESSorURL:122/monkeymike/MyGreatCodingProject.git

..or for http connections (using username and password – when pushing)

git remote add origin http://yourIPADDRESSorURL:180/monkeymike/MyGreatCodingProject.git

then push to it with:

git push -u origin –all

If I want to connect to the docker container over SSH I use:

ssh git@myserverURL -p 122

..and when I connect to the server, front end, I use http://myserverURL:180 and it’s all good.

NuGet on Rider C# IDE from JetBrains

You can find NuGet under the Tools menu, then select NuGet and then Manage Nuget Packages.

Once you have the packages dialog open type in the name of the package, choose the package you want and use the green cross button (on the right hand side) to add it to your project.
That should add it to your project; nice and easy.

Wales Pharmacy Data. Load unto mariadb.

You can download some data from here.
Unzip the files into your data directory. The GPDataYYYYMM.csv files are the ones you want.
This assumes you have mariadb installed and all working OK.

Unfortunately the structure of the csv files changes between 2013 to 2015 has one set of columns and 2016 onward have another set.
as per Blue Peter I’ve created a database called pharmacy before hand.

create table pharmacy.GPData_OLD
(
	HB text null,
	Locality text null,
	PracticeID text null,
	BNFCode text null,
	BNFName text null,
	Items int null,
	NIC double null,
	ActCost double null,
	Quantity int null,
	Period int null
);

The new structure has a couple of extra columns: DDD and ADQ.

create table pharmacy.GPData_NEW
(
	HB text null,
	Locality text null,
	PracticeID text null,
	BNFCode text null,
	BNFName text null,
	Items int null,
	NIC double null,
	ActCost double null,
	Quantity int null,
	DDD double null,
	ADQ double null,
	Period int null
);

Now we’ve created the tables we can use a bash script to load all the csv files into the tables we’ve just knocked up.
This from something pulled from Stack Overflow:

for f in /DATA/mysql/pharmacy/GPData201[345]*.csv
do
    mysql -e "LOAD DATA INFILE '"$f"' INTO TABLE  GPData_OLD FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES" -u root --password="yourrootpassword" pharmacy
echo "Done: '"$f"' at $(date)"
done

save it as something called “something” and then run it by invoking like ./something. Remember chmod u+x something

for f in /DATA/mysql/pharmacy/GPData201[67]*.csv
do
    mysql -e "LOAD DATA INFILE '"$f"' INTO TABLE  GPData201706 FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES" -u root --password="TreeOfLife123$" pharmacy
echo "Done: '"$f"' at $(date)"
done

as something like “something2016” invoking with ./something2016 again make it run-able using chmod u+x something2016

Chug chug chug and you get something like 33 million rows.

Moving Files By Name

…where the name is made up of the date and time.

I’ve got a lot of image files that I tend to dump into whatever directory I feel like.
This simple python script will create directories and move files around based on the file name, as long as the filename follows the following format YYYYMM. Bascially as long as the first 6 characters are integers it should work. In my file system that’s fine In yours the same may not be true. If the first 6 characters is not an integer then it will leave them alone.

Notice I’m not checking for JPG/MP4 files only; I’m not checking that the year or month are valid values; I’m not checking the EXIF values and basing it on them if the filename is not valid; there are lots of really cool things I could do as well but I don’t need those because that’s how my files are set out. I’m not building in lots of features that don’t touch my use-case.

import os
import shutil

path = '/home/mike/TestPython/'
os.chdir(path)

for file in os.walk(path):
    for name in file[2]:
        year = name[:4]
        month = name[4:6]
        if name[:6].isdigit():
            if not os.path.exists(year):
                os.makedirs(year)
            if not os.path.exists(year + "/" + month):
                os.makedirs(year + "/" + month)
            src = file[0] + "/" + name
            dst = path + year + "/" + month + "/"+name
            shutil.move(src, dst)

Once you’ve moved all the files into their new homes delete all the empty directories. This code has been ever so slightly modifed from this StakOverflow post. It was modified for Python 3, by adding brackets to the print syntax:

import os
currentDir = '/home/mike/TestPython/'

index = 0
for root, dirs, files in os.walk(currentDir):
    for dir in dirs:
        newDir = os.path.join(root, dir)
        index += 1
        print (str(index) + " ---> " + newDir)
        try:
            os.removedirs(newDir)
            print("Directory empty! Deleting...")
            print(" ")
        except:
            print("Directory not empty and will not be removed")
            print(" ")

Finding duplicate image files.

I love my phone. Having it means that I am rarely without a camera to take pictures of inane and awesome stuff. However I have very bad habits end up copying them more than once and putting them into all sorts of funny places. As a result my “Pictures” has a lot of duplicates. To clear up the duplicates I use a program called FSLINT.

PostgreSQL access over the local network.

After installing postgresql

Edit the pg_hba.conf
usually

/var/lib/pgsql/data/

under the line

# TYPE DATABASE USER ADDRESS METHOD

comment out all the lines using #
and then add

host all all 127.0.0.1/32 trust

and

host all all 192.168.0.0/24 trust

and that seems to work. I can log in to the server over my local network using DataGrip.

Firefox: Play BBC New videos without installing flash.

Go here: https://unop.uk/dev/how-to-watch-bbc-news-videos-on-a-desktop-without-flash/

and here: http://www.howtogeek.com/113439/how-to-change-your-browsers-user-agent-without-installing-any-extensions/

basically create a new item in about:config called general.useragent.override and paste

Mozilla/5.0 (Linux; Android 4.4.4; en-us; Nexus 5 Build/JOP40D) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2307.2 Mobile Safari/537.36

into it.