Mac OS X "Open"

I just love the OS X command 'open'.  No matter where you are you can just type open filename and it'll figure out the associated app and open the file with it.  I wished there was a Linux equivalent 'till go figure... there is one.

I found this stackoverflow post mentioning the command xdg-open.

that's a bit convoluted to write, so i made an alias

alias lopen='xdg-open'

lopen foobar.pdf
lopen .
lopen somefile.py

All of it works flawlessly.  :D I love it

PS. open command is taken by an openvt binary.  Hence lopen.

Hint:  To change the default association, in gnome at least:

Right-click->Properties->Open With->Set as default (lower right).

If you need to add a custom application. (I'll assume you created a .desktop icon and it shows up properly in your gnome menu using alacarte or by hand)

You need to update your:

 ~/.config/mimeapps.list

For example in my case, I added the following line:

text/plain=alacarte-made.desktop;

where alacarte-made.desktop is my sublime editor.

You can fine a list of most of the supported mimetypes in:

/usr/share/applications/defaults.list

Fixing Font Rendering In Chrome Fedora

Fonts in general aren't as pretty in Fedora out of the box as they are in Ubuntu.  But the Chrome rendering is horrible.  I had to add this little gem to my ~/.fonts.conf file.

<match target="font">
 <edit name="autohint" mode="assign">
    <bool>true</bool>
 </edit>
 <edit name="hinting" mode="assign">
    <bool>true</bool>
 </edit>
 <edit mode="assign" name="hintstyle">
    <const>hintslight</const>
 </edit>
</match>

Ubuntu 15.04 Broken Upstart / Working Systemd

Ubuntu 15.04 I believe has switched over to upstart which in turn has broken more then a few 3rd party apps I had installed.

For example whenever I tried to install plex media server I would get the following error:

"initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused"


Here's the fix that worked for me:

sudo dpkg-divert --local --rename --add /sbin/initctl
ln -s /bin/true /sbin/initctl

Credit for this little tweak: https://www.nesono.com/?q=node/368

Raid 5 to ZFS (Part 3 )

I posted a few articles in the past on migrating from Raid 1 to Raid 5. This will be my attempt to migrate from a raid 5 system to ZFS.

Setup

sudo apt-add-repository --yes ppa:zfs-native/stable
sudo apt-get update
sudo apt-get install ubuntu-zfs

Validate that the module was load.

sudo modprobe zfs
sudo dmesg | grep ZFS

Expected output should be something along these lines:

[  824.725076] ZFS: Loaded module v0.6.1-rc14, ZFS pool version 5000, ZFS filesystem version 5

This should get you all you need to create a standard ZFS setup. If you're starting from a clean slate, follow this guide. I'll follow it as a guide, though my plan is to shift from Raid 5 to ZFS. (Naturally backup, backup, backup....oh and did I mention backup? Okay good).

Mesos on Ubuntu 13.10 Part 1

This is my experiment to get mesos setup on Ubuntu.  Think of mesos as an layer running on top of your native OS that manages the machine's resources.  You can then request and allocate resources as needed.  This is just a blog post where I went through the steps and got master setup to run on an Ubuntu Server.

Install instructions come from:  http://mesos.apache.org/gettingstarted/

Install Dependencies.

sudo apt-get update
sudo apt-get install build-essential openjdk-7-jdk python-dev python-boto libcurl4-nss-dev libsasl2-dev autoconf libtool 

Build Mesos

    # Change working directory.
    $ cd mesos

    # Bootstrap (***Skip this if you are not building from git repo***).
    $ ./bootstrap

    # Configure and build.
    $ mkdir build
    $ cd build
    $ ../configure
    $ make -j

    # Run test suite.
    $ make -j check

    # Install (***Optional***).
    $ make install

Examples

 
# Change into build directory.
 $ cd build

 # Start mesos master.
 $ ./bin/mesos-master.sh --ip=127.0.0.1

 # Start mesos slave.
 $ ./bin/mesos-slave.sh --master=127.0.0.1:5050

 # Visit the mesos web page.
 $ http://127.0.0.1:5050

 # Run C++ framework (***Exits after successfully running some tasks.***).
 $ ./src/test-framework --master=127.0.0.1:5050

 # Run Java framework (***Exits after successfully running some tasks.***).
 $ ./src/examples/java/test-framework 127.0.0.1:5050

 # Run Python framework (***Exits after successfully running some tasks.***).
 $ ./src/examples/python/test-framework 127.0.0.1:5050

Cloud Security Encryption

I've started using Google Drive under Linux with the automatic sync being provided by InSync.  One of my concerns, like most people is security.  Like with any security system, the more security it is the less convenient it is.  You gain more security by sacrificing more conveniences.

ie.  The most secure storage, is a turned off computer put in a safe.  (it's secure and utterly useless).

My first approach is a bit less secure, but fairly convenient.

1.  InSync basically lets me sync files to supposedly multiple Google Drive accounts, though I still have to figure out the exact procedure for that.  For now, I drop a file in $HOME/Insync/account_name/ and it magically appears in my google drive.  Some files, say bills, and tax records I might not want to put on my google drive.

Approach A

I decided to keep all my files as pdfs.  So, all I need to do is figure out a way to encrypt them.  I decided to use pdftk which is pretty much the best tool kit I've found for manipulating PDFs.

Here my steps:

1.  Scan / place file into a non-cloud enabled location.  Let's call this $HOME/tmp/ and call the file open.pdf

2.  Encrypt the file.

     pdftk $HOME/tmp/open.pdf output encrypted.pdf owner_pw s3cr3t

     naturally, replace s3c3t with your own password.

3.  move file to your synchronized 'folder' where it will be kept.

Advantages:

• Qausi secure.  128bit encryption on pdf files.
• Google understands encrypted files, and will prompt you for your password when trying to open your pdfs.  Which means, you can open and view your secure files via any web browser.

Disadvantages:

• 128Bit encryption is inconvenient to break, but not impossible.  If you're storing important information.. like say your tax records with your SSN, you probably want something more secure.

Approach B

This approach was intended to take advantage of boxcryptor.  I was following an old guide for this service to allow for linux support, though right now, I really can't see a reason to use them, since as far as I can tell, a native Dropbox/ Google Plus client + encfs will provided essentially the same benefits.

Directions:

1.  We'll be relying on encfs.  I think cryptkeeper is supposed to be a nicer more user friendly way of doing this, but I could figure out how to use it, or cared enough to look into it, so we'll be doing this the CLI way.

sudo apt-get install encfs

sudo apt-get install cryptkeeper #optionally 

2.  we'll call your sync folder "cloud", replace this with ~/Dropbox, ~/insync/username/ or whatever is appropriate in your case.

cd ~/cloud

mkdir ~/temp_cloud

mv ~/cloud/* ~/temp_cloud  # moving all files out of cloud storage

3.  This will probably delete all your 'cloud' storage, but that's okay.  We just moved the files out into a local folder.  If you're doing this with gigs and gigs of data...this might take a while.

4.  Mount your ~/cloud as an encrypted folder

encfs ~/cloud ~/secure_storage  #I just used p for default settings and entered a nice long password.

mv ~/temp_cloud/* ~/secure_storage/

5.  Enabling automatic mounting of ~/cloud 

sudo apt-get install  libpam-mount

sudo gpasswd -a USERNAME fuse

sudo vim /etc/security/pam_mount.conf.xml

insert a line resembeling this one right below:

<!-- Volume definitions -->

        <volume user="user" fstype="fuse" path="encfs#/home/user/secure_storage" mountpoint="/home/user/cloud" />

6.  Reboot, and your drive should be automatically mounted.  Now IF you want to ensure that the file is encrypted then you should only be working in the folder ~/secure_storage.  Any files you create in ~/cloud directly will be plaintext and in-secure.

Advantages:

• Much higher lever of security then anything a PDF crypto will provide.
• Pretty much transparent.  You could also configure your cloud app to look at ~/crypto and mount ~/Dropbox (for example).  essentially switching the order.  

Disadvantages:

• All the files are utter garbage via a web browser.
• Main big issue I have right now, is there seem to be a file name limit.  You can't have a file name longer then 36 characters.  

References

1. http://pragmattica.wordpress.com/2009/05/10/encrypting-your-dropbox-seamlessly-and-automatically/   (Prettier version of Approach B I found while looking up stuff as I was writing this document) 

Social Media History Clean Up

I'm not sure if anyone would find this useful, but made a very green alpha release of a pet project I had for a while.

• the release file can be retrieved here (Google's Drive). 
• Project website:  https://github.com/safaci2000/SocialMediaCleanUp

I was getting annoyed with twitter and other social media when I wrote this.  I mainly wrote a social media finger print cleanup.  It asks for permission and will flush out your entire twitter minus two weeks delay.  

So if I ran this today, 12/19/2012, then any post I made that was on or before 12/04/2012 would be deleted. 

I know it needs to be a lot more configurable .and I have several todo items associated with this project.  I'm just curious to know if anyone would find it useful.

Netflix On Ubuntu

Disclaimer: Full credit for this article goes to:  http://www.omgubuntu.co.uk/2012/11/how-to-use-netflix-on-ubuntu  This is nothing but a repost.

I usually post random tech items and a few other oddities that interest me...and netflix coming to the Linux Desktop has me super excited.

1.  Install Netflix Desktop

• sudo apt-add-repository ppa:ehoover/compholio
• sudo apt-get update && sudo apt-get install netflix-desktop

2.  start netflix-desktop and follow the screen prompts.  It should download the gecko engine, install silverlight etc.... and you should be able to stream your netflix videos.

If you do get this working.. I am curious to see what the performance is for you.  It looks like it runs.. but it's definitely not native.  A step in the right direction though....

On a side note.  There is this big project spiraled by Ubuntu where they're trying to get Ubuntu running on an android phone.  I wonder if the opposite would be possible, and if we'll ever see a native Android app run on an Linux Desktop.  (Letting me run my android netflix app on my 24" LCD monitor ) 

Changing Default / Primary Monitor in Gnome 3

The proper way to do this is to run System Settings -> Display and drag the black bar at the top of your primary monitor to another monitor.  I find the Display interface really difficult to use and unresponsive.

So, as much fun as it is to try to get that UI to work, this is my work around using the CLI.

list all your monitors:

xrandr  

This will list the IDs of all your current monitors.  To set a different monitor as the default type something along these lines:

xrandr --output HDMI3 --primary

where HDMI3 is the ID of the monitor you want set as your primary monitor.

Converting from Raid 1 to Raid 5

This assumes you have a functional Raid 1 and wish to convert it to a Raid 5.

Disclaimer:  At some point during this process I realized that I had a bad mother board.  The reason my /dev/sdd1 failed wasn't the drive, but the bus on the board.  That being said, this is the unverified procedure.

I'm running on Ubuntu 12.10 but you should be able to do this on any modern Linux distribution.

1.  Stop all mdadm related tasks.

sudo umount /media/data #of whatever mountpoint is appropriate.
sudo /etc/init.d/mdadm stop 
sudo mdadm --stop /dev/md0

2.  Change the raid layout

This part is kind of scary, and I wouldn't advice mounting the raid at this point.  I especially didn't like the fact that it looks like it's overwriting my raid.. that made me nerveous, but it's essentially restrucuting how data is stored, and putting  a raid 5 mapping on 2 drives.  ie.  creating a degraded raid 5.


#update this as appropriate.
mdadm --create /dev/md0 --level=5 --raid-devices=2 /dev/sda1 /dev/sdd1 

WARNING: wait for this step to complete.. look at /prod/mdstats and wait for it to finish before proceeding.



3.  Add a 3rd drive.

#in my case I started with /dev/sdd1, added /dev/sda1 to create a raid 1, and then adding /dev/sdb1 as the final device.  You don't have to follow my convention.  That made sense for my use case, since my dead drive was /dev/sdd you can simply start with /dev/sda1 and go alphabetical.

mdadm --add /dev/md0 /dev/sdb1
mdadm --grow /dev/md0 --raid-devices=3

This part took around 15 hours for me.  

At this point, I'd be okay with mounting my raid partition.  Again, it's safer not to... but.. it's won't break the process if you do.

4.  Expand File System.

At this point what we have is the equivalent of have a large hard drive, but a smaller partition on it.  We need to grow the local file system.

I'm covering 3 use cases.

a.  LVM

I've have had lvm on my raid before.  Actually, I used to have raid + lukefs encryption + lvm.  Too many layers though the performance isn't as bad as you might expect.

TODO:  I have to look this up... I'll update this eventually.



b. XFS

xfs is a bit odd, you need to have the file system mounted in order to grow it.

xfs_repair -n /dev/md0  #just to be safe.
mount /dev/md0 /media/data

xfs_growfs /media/data


c. EXT3/4

e2fsck -f /dev/md0 #check file system
resize2fs /dev/md0   #grow file system

5.  Update fstab

This should not need any changes, but just in case:


/dev/md0        /media/data     xfs     defaults

6. Update mdadm.conf

sudo su - 
mdadm --detail --scan >> /etc/mdadm/mdadm.conf

##edit the file and remove the next to last line.  ie.  The command above appends the new mdadm config to your config file.  So remove the previous raid 1 line.  There should be a single line defining md0 which looks something like this:

ARRAY /dev/md/md0 metadata=1.2  UUID=0ec3c5aa:5cee600b:ef1e8f7d:09b20cc8

This is the line I removed:

#ARRAY /dev/md/md0 metadata=0.90 UUID=bf8a2737:554e654c:c2eab133:b01f9710

In other words, assuming you only have 1 raid setup, your mdadm.conf should only have a single ARRAY configured.


References:  

1. http://www.arkf.net/blog/?p=47
2. http://www.davelachapelle.ca/2008/07/25/converting-raid-1-to-raid-5/

Raid Upgrade Trick

Premise:  Here's my situation.  I have a raid 5, with 3 1 TB drives, which gives me about 1.8 give or take of storage.  I had one drive go bad on it, and I found a great deal on amazon, so I decided to upgrade to a new raid 5 configuration with 3x 2 TB drives.

To recap.

Current setup:

• 3 drives
• 1 TB of space for each
• Raid 5
• total raid capacity:  around 1.8
• space used:  1.3 TB

New Setup:

• 3 drives
• 2 TB of space for each
• Raid 5
• total raid capacity once completed:  around 3.8 TB

Problem:

1.  I don't have enough ports on my machine to plugin all of the drives for my 2 raids.  Ideal would require 6 sata ports for the 2 raids and a 7th one for my OS drive. Even if I futz with things, and allow for one drives to be missing from each.  (ie. run a degraded raid), I would still need 5 ports, which I don't have. 

Solution:

Step 1.  Copy data

Note:  This can be done a bit differently...but I think my approach will save you time in the long run. 

I replaced the dead 1 TB drive with one of the new drives.  The computer starts up, It brings up the old raid /dev/md0 in a degraded state per usual, and I now have 1 new drive to play with that I know is good.

Using the new drive, I'm creating a new degraded raid 1 device.

mdadm --create /dev/md1 -l raid1 -f -n 1 /dev/sdd1

mkfs -t xfs (or whatever you prefer) /dev/md1 

sudo mount /dev/md1 /media/newRaid

rsync -avP /media/oldRaid/ /media/newRaid/    ## be careful the / matter and change rsync's behavior.

So at this point, we have 2 raids up and running, and I'm copying data from the old raid to the new one. 

Step 2.  Replace Drives

At this point, shut down the machine, remove your remaining drives from the old raid, and replace them with the new one.  comment out your /etc/fstab if you're mounting your raid by default.

It will probably create some random /dev/md* depending on your OS and intelligent it is.  Ubuntu re-created the raid array as /dev/md127, which I'll use for now.  we can easily rename it to md0 once everything is done.

Let's add another drive to the array.

mdadm --grow /dev/md0 -n 2  
mdadm --manage /dev/md1 --add /dev/sda1

If the above command works, then if you issue a 

cat /proc/mdstat 

You should see data being synced.

VERY IMPORTANT... wait for the sync to finish before doing anything else at this point.  

This gives me a new functional raid 1.  We're not done.  The next step is to convert a raid 1 to raid 5, and since I still have 2 functional 1 TB drives where I can reconstruct my old raid with, I also have a backup of the data.. so if I lose everything.. I just lose another day or so to copy data back.

Side note:  ETA to copy 1.1 TB of data was about +8 hours.  I started around 6 pm and it was still going at 2 am.

ETA to do the sync above is estimated at 4-5 hours.  Though thankfully I can leave my machine up and running and use it while it's synching.

(I forgot to mention, if you want to access your data while it's synching.. you can mount the drive:

sudo mount /dev/md127 /media/data  

naturally it's recommended not to use your computer while doing that.. but meh... next step will take even longer... (Raid 1 to Raid 5 conversion).  I was actually playing a game while I was copying the data (rsync).  Though I did re-run the command once more once everything was done to make sure it got everything.

Look forward to the next article while will talk about the raid conversions.

References:
 1.  Create A Degraded Raid 1 Array

Back in town.

Moving back to the city.  I'm excited.  That is all.

Migrating from Mailman to Google Groups

Migration Process

I've recently migrated a mailing list from a server running postfix + mailman to a google group with the idea
of ease of management and removing the need of having a physical server.

Someone asked me to comment on my thoughts on the migration via email, so I'm mostly sharing the same info for anyone who might have an interest.

-----
I retrieved the full listing of all emails from mailman via the CLI
> ./list_members <listname> > members.txt

Then login to the google groups interface paste the file in and invite all your members
to join the new mailing list.  I made the list moderated and set an autoreply to all incoming messages telling the members to join the google group.

I waited about a week or two for people to join and such... then I changed the DNS entries and moved the mail server to a google hosted domain.  I recreated a user with the previous list name...

so if my mailing list name was   javaFanBoys@foobar.com  I created a new user with that same
address and created a vacation message instructing the user that the old email list has moved over.

In my case I had an announce list for events that forwarded all its emails to the more general list.

I haven't figured out how to make it broadcast to all users though there is an archiver email that takes whatever email it receives and treats it as if it was a regular email to the list in all regards except the fact
that it doesn't send it out as an email to the user base.  (ie visible via web view but not via email)

Under your groups advanced setting you'll have an email along these lines:  listname+garchive-###@googlegroups.com.  So I created a new account again named bot, which is subscribed to both lists, and any emails received from my -announce list gets forward to the email address listed above.

Motivation


The reason I decided to move over was for:

1. easier management and delegating admins/moderators and such
2. removing required infrastructure
3. migrate the main website to sites.google.com 
4. consolidated callendar of events of various associated groups in the area 

I don't like having to rely on a single person who has the master password to make any major changes.  The delegation of rights via google is a lot easier.  You can setup email distribution groups like info@domain or help@domain to allow everyone to be involved and informed.

Problems

Here are the current limitations of google groups.

1. There is no easy way of importing old posts short of re-emailing all the history to the list. 
2. It's difficult to forward emails from one group to another and have it be broadcasted to users.
3. sites.google.com is great for simple content, but if you want to edit html or do anything remotely fancy it is very limiting.

Create a Windows Boot Disk from Linux

I tend to not have any Windows machines around unless I need them, and I'm getting tired of the unlabeled CDs floating all around me.  I decided to see if I could figure out a way on how to make a bootable windows USB thumb drive from Linux.

initial solution can be found here

1.  format your usb thumb drive to NTFS.
     sudo mkfs -t ntfs /dev/sdb1
2.  make the partition bootable:
     #fdisk /dev/sdb
      a (toggle bootable flag)
     1 (select partition I'm presuming you only have a single partition )
     w (write changes)
3.  mount your cdrom (which should happen automatically ) or the ISO.
      # mount -o loop win7.iso win_disk
4.  mount the usb drive.
     # mount /dev/sdb1 /mnt/usb
5.  copy data over.
     #rsync -avP /mnt/cdrom/  /mnt/usb/
6.  Download and build ms-sys which you can get from here:
7.  Make USB thumb drive bootable
     # ms-sys -7 /dev/sdb

Dynamic Free DNS

I used to use everydns.net which was later acquired by DynDNS.  Sadly, once acquired they started migrating users from the free service to their paid subscription service which as far as I can tell all it provides
is the ability to point foobar.com to a dynamic IP, as well as modifying your DNS records (A, CNAME, MX, etc ).

Once alternative is zoneedit.com but my domain seems to be tied into a weird flux having used them in the past that I couldn't use their service as an alternative, so I ended up setting up a custom solution that is probably overkill for most users, but it might be interesting information.

So, I need to point my domain to have 2 name-servers that manages my zone.

I only care about 1 zone really, in my test case I just pointed my 2nd name server to the default my registrar uses.  So, if my primary DNS is down, I'll probably get the standard this page is registered and is owned by foobar.com.

The main issue is that I need to point a registrar to a dynamic IP.  So I setup no-ip service to auto update.  I pointed my Domain registrar to a no-ip address and go figure it accepted it.

So now I can point it to my own IP that can potentially change and dns resolution will go to the proper address.

Now, naturally you need to expose port 53, and you need to run a DNS server of your choice I went for bind.

Now, I can have an CNAME record in my dns that points to say foobar.no-ip.com but all that would give me is a redirect, and I wanted to have my own domain after all.

I need to setup at least one A record that maps to my public IP.  This is easily done, and usually looks something along these lines:

www           IN        A       4.4.4.4

with 4.4.4.4 being your public IP.  The problem that arises is that even though no-ip autoupdates to point outside requests to my DNS, unless I updated bind to point to the right address then there is no point.

My hacky solution was to write this python script that takes a list of bind files to update.

It'll read my dns zone file, search for any IPs not matching my LAN subnet, and update all those records with the current IP address.  Once it finishes, it'll reload bind to make it read the new updated config.

current code is in:  https://github.com/safaci2000/dns_utils though like I said its hacky.

It establishes if an IP is local or not by comparing the first octet.  I need to add some logic that actually checks if the IP is public or not.

So..once this all done I have a domain that points to foobar.no-ip.com for its dns host.

foobar-ip.com points to my machine as long as it's online.  Then I have an hourly updated python script that will update and reload my DNS records.  There is the issue that if my DNS server goes down, nobody
can access my machine.  This normally wouldn't be an issue (for me) since my DNS all points to the local machine, but I am hosting my email with google.  So if my dns server goes down, in theory I could
be losing mail.  I was thinking of just getting a VPS and setting this up a secondary machine, but if I get
to the point of paying $20/mo for a machine to run a DNS zone, then I might as well just pay dyndns my monthly fee.  Then again, a full VPS would be more useful to me and could have some more potential features then just a simple dns updater which is essentially recreated here.

Thoughts, comments?

Making Sense of Eclipse....

Oh Eclipse.. how I love you and how I loath thee.

On linux, I probably think that eclipse is one of the most versetile IDE environments I've ever used.  Pick your language and it probably supports it.  Java, pyhon, c++, cobol as well apparently for those wanting to dust off their mainframe leet skills. 

The main problem I've had with eclipse for ages has been their plugin support.  Yes, they have tons of plugins and most of them work wonderfully.  I start out with the bare eclipse, then I have a need for a python environment.. so I install the plugin, then I need a c++ environment so I install that plugin. 

Then I realize that I have so much clutter in my IDE that I can't find anything.  I also think that if I'm working on a c++ project, I most likely don't want, care or need the python support in my IDE.  Its only purpose is to suck even more of my precious RAM.

I hate having so much clutter.. so I usually end up with 3-4 installs of eclipse which is really silly.

Here's my attempt at fixing this.  (yes, this is super nerdy but it mostly works).

cd $HOME/local; mkdir eclipse
 
extract the latest eclipse into that directory, and add your standard language
neutral customization.
I like using the viPlugin or eclimd and the colorthemeplugin.


Standard install, so extract your eclipse, setup your plugins.. then go to your eclipse install directory (I usually install in $HOME for my own sanity if nothing else)

cd eclipse; git init; git add * 

Also include any dot files in the eclipse directory.

git commit -a -m "initial install"; git tag bare

now, let's setup our java environment.

git checkout master -b java

Get your favorite java plugins.  Maven?  GUI Builder? whatever you like using.

Add everything that's been added to the repo.

git commit -a -m "Adding java stuff"

Now, let's do the python environment.

git checkout master -b python

Install the python plugin and any tools.

Once this is all setup...  you checkout the branch you want to work on different versions of eclipse that are slimmed down. 
You can checkout the bare install at anytime (git checkout bare -b newFork).

In theory it saves you some space, and lets you get somewhat of a handle on your environment.

Well.. that or its a completely random crazy idea that makes no sense to anyone but me.... but sharing anyways.

Thoughts? Comments?

Also, if anyone knows of a way to unload plugins without uninstalling them completely, please let me know.

An interesting but mostly pointless exercise.

So, let me explain my mind frame when I was approaching this.  Some of it is quasi logical, other parts is just me being weird.

For some reason or other.. I wanted to have a few raw block devices.  They're usually created using dd command.

ie.  dd if=/dev/zero of=foobar.raw bs=1M count=2048
bs = block size so the size of foobar.raw = 1m * 2048 = 2048mb = 2 gb.

Now some operations you can perform directly on the file foobar.raw (like formatting it), but since I'm trying to do crytpo + lvm, I need to at least pretend that it's a device.  losetup lets me associate a device name, in my case /dev/loop0 with a file on my file system.

So...  here's my instruction set to get a file to behave like a file system, and allow you to do all sorts of unholy things to it.

**Most of these commands assume root.  Either prepend sudo to all of these, or just become root via su - or your favorite procedure to get god mode.

1.  Create a raw file.

    # dd if=/dev/zero of=foobar.raw bs=1M count=2048

2.  Losetup, to associate file with a device name.

    # losetup /dev/loop0  ./foobar.raw


3.  Cryptosetup, really really not needed, but hey.. why not. 

3a.  Format the device we just created (loop0) to be cryptsetup device.

cryptsetup -y --cipher aes-cbc-essiv:sha256 --key-size 256 luksFormat /dev/loop0

3b.  Lets open the device, and give the encrypted device a name. 

 cryptsetup luksOpen /dev/loop0 cryptoLoop    ## 

4.  Now that we have an encryption layer.. lets's create an LVM, which allows me to 

extend the size of my file system or join multiple files...and all sorts of craziness/flexibility.

Most of these steps are your standard LVM setup, but we'll note them here for consitency.

pvcreate /dev/mapper/cryptoLoop
vgcreate cryptoLvmGrp
lvcreate -l100%VG -n lvmData cryptoLvmGrp

now, in theory you can format the lvm.

mkfs -t ext3 /dev/cryptoLvmGrp/lvmData
mount /dev/cryptoLvmGrp/lvmData loopback

standard usage rules apply, permissions and such.  Once you're done
using it, you need to shut down all the layers in order.

Shutdown:

1.  umount loopback

2.  vgchange -a n 

3.  crypsetup luksClose cryptoLoop

4.  losetup -d /dev/loop0 

So, in retrospect. I can probably just create an xfs/ext file system and expand the file system as needed... but meh.. I kinda like this elaborate setup.

I'm going post some scripts to automate the creation, loading, and unloading of these systems, though some of the paths for LVM and cryptsetup depends on the distro.

Edit:  https://github.com/safaci2000/RawCryptoLVM  Some really dumb scripts that work for me.  

Gentoo is Amazing!!

This is a flame post on Gentoo.  I found it funny mainly because I used Gentoo for the greater
part of Linux experience.  I really think it's an amazing distribution.  That being said.. enjoy the laughs.

http://funroll-loops.info/

HP Mini 210–1076NR + Ubuntu Netbook 10.10

I had a hell of a time getting the broadband working.  This particular devices comes with
Gobi 2000 and I had to do a lot of silly playing around to get it up and running.  It is odd
that the live CD out of the box supports it, but the final installed version didn't.

If you plan doing an install, I would try to be somewhere you can get a Ethernet cable to
plugin so you can get all these updates and get things squared away.

1. if it's not installed already, (apt-get install gobi-loader) get the firmware from the windows partition.
There should be a folder named Qualcomm in your standard Program Files folder.  Which contains the various firmware.  This document explains which folder to get:.  You need to grab the content of the appropriate folder
and copy it to /lib/firmware/gobi

reboot and everything worked fine for me.  If that fails, you may have to load it manually.
    gobi_loader /dev/ttyUSB0 /lib/firmware/gobi

fyi.  If you have the wrong firmware (at least in my case) it just hanged there indefinitely, while when I had the right firmware it worked almost instantly.

Wireless didn't work for me out of the box, but that's the typical broadcomm being "special" thing.  If you do the usual updates it started working for me again due to penguin magic from the ubuntu repos.   I'm sure there's about 800 sources of how to get this working, so I won't go into it.  If you have trouble.. post a comment, and I'll update the article.

Conversion to Linux Test Case.

Part 0 -- Intro

So, a friend of mine inquired about the joys of Linux, and had a genuine curiosity about the operating system.  I gave up trying to push Linux on people in general.  I find it cause me more headaches then its worth, and if people don't want to learn, no matter how amazing and awesome I think Linux and penguins are it still involves people using something different.  I tend to find that the general population, no matter their intellect and abilities are petrified of change.

Part 1 -- Poor Guinea Pig.

My friend is an art student, with an aptitude for computers and technology.  She won't be pulling up the terminal and starting some gcc, and patching the kernel, but she's capable of basic common sense and critical thinking.  She's worked in a computer lab before.... so I'd call her a power win user? somewhere along those lines.

She loves her shiny android, and nook, and just recently discovered that they're Linux based and still marvels at all the tidbits of Linux hidden away in various technologies.

I personally have been using Linux for so long, that I've lost touch with what's easy and isn't, so I thought she'd be a good use case.

Part 2 -- Target machine 

She just recently purchase a Netbook from verizon, with Windows 7 start edition.

Windows 7 Starter issues:
1.  limited to 3 concurrent process at a time.
2. Can't change the wallpaper.

Requirements:
Office of some form.  (OpenOffice/google docs worked)
IPod (Banshee /Rhythmbox seemed to work )

I had her look at Linux Mint, Ubuntu Netbook edition, and Fedora...and she liked the Unitiy look.

I tried walking her through an install, and that failed miserably once the idea of partitioning, mount points and all that came up, so instead decided to do it in person.

Netbook:  HP Mini 210–1076NR

Live CD issues.  Wireless didn't work.   Possible mouse issues.

I partitioned the machine proper using gparted and set it up to dual boot windows 7 and Ubuntu.  I went through the install process, everything seemed to work fine once it was done with a few exceptions.

Wireless was still broken, but that's okay.  We had the Verizon broadband, went online.. had to do some manual tasks to get the firmware download and get the silly wifi working.  First time using unity, so it didn't click right away, but the mouse won't right click.  (xorg/touchpad config probably).

I got her wifi updated, then had her do the updates she was prompted for.  The updates broke her Verizon broadband.and I tracked it down to a bug:

** That's a big no-no.  Ubuntu is supposed to be super user friendly... once you make a release, you can't break drivers by doing an update.  Especially when the fix is patch the kernel and recompile... or go manually find the firmware for your card.

This worked out of the box... it should not be this difficult for an end user to get things fixed.  If the target audience is ease of use, intricate command line tools, patching code, reloading kernel module.. is not a proper way of handling things.

3.  Final thoughts.. so far.

So, I seemed more annoyed with Ubuntu behavior then she was.  She seems very happy with speed/performance over windows 7.

Enjoys all the random apps that free and available to use.  I have to fix her mouse still, but aside from Verizon and a bit of rocky start everything else works, but it involved more techie experience then it should have.

Many of these things should just work out of the box.


PS.  I need to track down the exact bug number, I'll link it in an update.