Social Media History Clean Up

I'm not sure if anyone would find this useful, but made a very green alpha release of a pet project I had for a while.

• the release file can be retrieved here (Google's Drive). 
• Project website:  https://github.com/safaci2000/SocialMediaCleanUp

I was getting annoyed with twitter and other social media when I wrote this.  I mainly wrote a social media finger print cleanup.  It asks for permission and will flush out your entire twitter minus two weeks delay.  

So if I ran this today, 12/19/2012, then any post I made that was on or before 12/04/2012 would be deleted. 

I know it needs to be a lot more configurable .and I have several todo items associated with this project.  I'm just curious to know if anyone would find it useful.

Netflix On Ubuntu

Disclaimer: Full credit for this article goes to:  http://www.omgubuntu.co.uk/2012/11/how-to-use-netflix-on-ubuntu  This is nothing but a repost.

I usually post random tech items and a few other oddities that interest me...and netflix coming to the Linux Desktop has me super excited.

1.  Install Netflix Desktop

• sudo apt-add-repository ppa:ehoover/compholio
• sudo apt-get update && sudo apt-get install netflix-desktop

2.  start netflix-desktop and follow the screen prompts.  It should download the gecko engine, install silverlight etc.... and you should be able to stream your netflix videos.

If you do get this working.. I am curious to see what the performance is for you.  It looks like it runs.. but it's definitely not native.  A step in the right direction though....

On a side note.  There is this big project spiraled by Ubuntu where they're trying to get Ubuntu running on an android phone.  I wonder if the opposite would be possible, and if we'll ever see a native Android app run on an Linux Desktop.  (Letting me run my android netflix app on my 24" LCD monitor ) 

Changing Default / Primary Monitor in Gnome 3

The proper way to do this is to run System Settings -> Display and drag the black bar at the top of your primary monitor to another monitor.  I find the Display interface really difficult to use and unresponsive.

So, as much fun as it is to try to get that UI to work, this is my work around using the CLI.

list all your monitors:

xrandr  

This will list the IDs of all your current monitors.  To set a different monitor as the default type something along these lines:

xrandr --output HDMI3 --primary

where HDMI3 is the ID of the monitor you want set as your primary monitor.

Converting from Raid 1 to Raid 5

This assumes you have a functional Raid 1 and wish to convert it to a Raid 5.

Disclaimer:  At some point during this process I realized that I had a bad mother board.  The reason my /dev/sdd1 failed wasn't the drive, but the bus on the board.  That being said, this is the unverified procedure.

I'm running on Ubuntu 12.10 but you should be able to do this on any modern Linux distribution.

1.  Stop all mdadm related tasks.

sudo umount /media/data #of whatever mountpoint is appropriate.
sudo /etc/init.d/mdadm stop 
sudo mdadm --stop /dev/md0

2.  Change the raid layout

This part is kind of scary, and I wouldn't advice mounting the raid at this point.  I especially didn't like the fact that it looks like it's overwriting my raid.. that made me nerveous, but it's essentially restrucuting how data is stored, and putting  a raid 5 mapping on 2 drives.  ie.  creating a degraded raid 5.


#update this as appropriate.
mdadm --create /dev/md0 --level=5 --raid-devices=2 /dev/sda1 /dev/sdd1 

WARNING: wait for this step to complete.. look at /prod/mdstats and wait for it to finish before proceeding.



3.  Add a 3rd drive.

#in my case I started with /dev/sdd1, added /dev/sda1 to create a raid 1, and then adding /dev/sdb1 as the final device.  You don't have to follow my convention.  That made sense for my use case, since my dead drive was /dev/sdd you can simply start with /dev/sda1 and go alphabetical.

mdadm --add /dev/md0 /dev/sdb1
mdadm --grow /dev/md0 --raid-devices=3

This part took around 15 hours for me.  

At this point, I'd be okay with mounting my raid partition.  Again, it's safer not to... but.. it's won't break the process if you do.

4.  Expand File System.

At this point what we have is the equivalent of have a large hard drive, but a smaller partition on it.  We need to grow the local file system.

I'm covering 3 use cases.

a.  LVM

I've have had lvm on my raid before.  Actually, I used to have raid + lukefs encryption + lvm.  Too many layers though the performance isn't as bad as you might expect.

TODO:  I have to look this up... I'll update this eventually.



b. XFS

xfs is a bit odd, you need to have the file system mounted in order to grow it.

xfs_repair -n /dev/md0  #just to be safe.
mount /dev/md0 /media/data

xfs_growfs /media/data


c. EXT3/4

e2fsck -f /dev/md0 #check file system
resize2fs /dev/md0   #grow file system

5.  Update fstab

This should not need any changes, but just in case:


/dev/md0        /media/data     xfs     defaults

6. Update mdadm.conf

sudo su - 
mdadm --detail --scan >> /etc/mdadm/mdadm.conf

##edit the file and remove the next to last line.  ie.  The command above appends the new mdadm config to your config file.  So remove the previous raid 1 line.  There should be a single line defining md0 which looks something like this:

ARRAY /dev/md/md0 metadata=1.2  UUID=0ec3c5aa:5cee600b:ef1e8f7d:09b20cc8

This is the line I removed:

#ARRAY /dev/md/md0 metadata=0.90 UUID=bf8a2737:554e654c:c2eab133:b01f9710

In other words, assuming you only have 1 raid setup, your mdadm.conf should only have a single ARRAY configured.


References:  

1. http://www.arkf.net/blog/?p=47
2. http://www.davelachapelle.ca/2008/07/25/converting-raid-1-to-raid-5/

Raid Upgrade Trick

Premise:  Here's my situation.  I have a raid 5, with 3 1 TB drives, which gives me about 1.8 give or take of storage.  I had one drive go bad on it, and I found a great deal on amazon, so I decided to upgrade to a new raid 5 configuration with 3x 2 TB drives.

To recap.

Current setup:

• 3 drives
• 1 TB of space for each
• Raid 5
• total raid capacity:  around 1.8
• space used:  1.3 TB

New Setup:

• 3 drives
• 2 TB of space for each
• Raid 5
• total raid capacity once completed:  around 3.8 TB

Problem:

1.  I don't have enough ports on my machine to plugin all of the drives for my 2 raids.  Ideal would require 6 sata ports for the 2 raids and a 7th one for my OS drive. Even if I futz with things, and allow for one drives to be missing from each.  (ie. run a degraded raid), I would still need 5 ports, which I don't have. 

Solution:

Step 1.  Copy data

Note:  This can be done a bit differently...but I think my approach will save you time in the long run. 

I replaced the dead 1 TB drive with one of the new drives.  The computer starts up, It brings up the old raid /dev/md0 in a degraded state per usual, and I now have 1 new drive to play with that I know is good.

Using the new drive, I'm creating a new degraded raid 1 device.

mdadm --create /dev/md1 -l raid1 -f -n 1 /dev/sdd1

mkfs -t xfs (or whatever you prefer) /dev/md1 

sudo mount /dev/md1 /media/newRaid

rsync -avP /media/oldRaid/ /media/newRaid/    ## be careful the / matter and change rsync's behavior.

So at this point, we have 2 raids up and running, and I'm copying data from the old raid to the new one. 

Step 2.  Replace Drives

At this point, shut down the machine, remove your remaining drives from the old raid, and replace them with the new one.  comment out your /etc/fstab if you're mounting your raid by default.

It will probably create some random /dev/md* depending on your OS and intelligent it is.  Ubuntu re-created the raid array as /dev/md127, which I'll use for now.  we can easily rename it to md0 once everything is done.

Let's add another drive to the array.

mdadm --grow /dev/md0 -n 2  
mdadm --manage /dev/md1 --add /dev/sda1

If the above command works, then if you issue a 

cat /proc/mdstat 

You should see data being synced.

VERY IMPORTANT... wait for the sync to finish before doing anything else at this point.  

This gives me a new functional raid 1.  We're not done.  The next step is to convert a raid 1 to raid 5, and since I still have 2 functional 1 TB drives where I can reconstruct my old raid with, I also have a backup of the data.. so if I lose everything.. I just lose another day or so to copy data back.

Side note:  ETA to copy 1.1 TB of data was about +8 hours.  I started around 6 pm and it was still going at 2 am.

ETA to do the sync above is estimated at 4-5 hours.  Though thankfully I can leave my machine up and running and use it while it's synching.

(I forgot to mention, if you want to access your data while it's synching.. you can mount the drive:

sudo mount /dev/md127 /media/data  

naturally it's recommended not to use your computer while doing that.. but meh... next step will take even longer... (Raid 1 to Raid 5 conversion).  I was actually playing a game while I was copying the data (rsync).  Though I did re-run the command once more once everything was done to make sure it got everything.

Look forward to the next article while will talk about the raid conversions.

References:
 1.  Create A Degraded Raid 1 Array