Back in town.

Moving back to the city.  I'm excited.  That is all.

Migrating from Mailman to Google Groups

Migration Process

I've recently migrated a mailing list from a server running postfix + mailman to a google group with the idea
of ease of management and removing the need of having a physical server.

Someone asked me to comment on my thoughts on the migration via email, so I'm mostly sharing the same info for anyone who might have an interest.

-----
I retrieved the full listing of all emails from mailman via the CLI
> ./list_members <listname> > members.txt

Then login to the google groups interface paste the file in and invite all your members
to join the new mailing list.  I made the list moderated and set an autoreply to all incoming messages telling the members to join the google group.

I waited about a week or two for people to join and such... then I changed the DNS entries and moved the mail server to a google hosted domain.  I recreated a user with the previous list name...

so if my mailing list name was   javaFanBoys@foobar.com  I created a new user with that same
address and created a vacation message instructing the user that the old email list has moved over.

In my case I had an announce list for events that forwarded all its emails to the more general list.

I haven't figured out how to make it broadcast to all users though there is an archiver email that takes whatever email it receives and treats it as if it was a regular email to the list in all regards except the fact
that it doesn't send it out as an email to the user base.  (ie visible via web view but not via email)

Under your groups advanced setting you'll have an email along these lines:  listname+garchive-###@googlegroups.com.  So I created a new account again named bot, which is subscribed to both lists, and any emails received from my -announce list gets forward to the email address listed above.

Motivation


The reason I decided to move over was for:

1. easier management and delegating admins/moderators and such
2. removing required infrastructure
3. migrate the main website to sites.google.com 
4. consolidated callendar of events of various associated groups in the area 

I don't like having to rely on a single person who has the master password to make any major changes.  The delegation of rights via google is a lot easier.  You can setup email distribution groups like info@domain or help@domain to allow everyone to be involved and informed.

Problems

Here are the current limitations of google groups.

1. There is no easy way of importing old posts short of re-emailing all the history to the list. 
2. It's difficult to forward emails from one group to another and have it be broadcasted to users.
3. sites.google.com is great for simple content, but if you want to edit html or do anything remotely fancy it is very limiting.

Create a Windows Boot Disk from Linux

I tend to not have any Windows machines around unless I need them, and I'm getting tired of the unlabeled CDs floating all around me.  I decided to see if I could figure out a way on how to make a bootable windows USB thumb drive from Linux.

initial solution can be found here

1.  format your usb thumb drive to NTFS.
     sudo mkfs -t ntfs /dev/sdb1
2.  make the partition bootable:
     #fdisk /dev/sdb
      a (toggle bootable flag)
     1 (select partition I'm presuming you only have a single partition )
     w (write changes)
3.  mount your cdrom (which should happen automatically ) or the ISO.
      # mount -o loop win7.iso win_disk
4.  mount the usb drive.
     # mount /dev/sdb1 /mnt/usb
5.  copy data over.
     #rsync -avP /mnt/cdrom/  /mnt/usb/
6.  Download and build ms-sys which you can get from here:
7.  Make USB thumb drive bootable
     # ms-sys -7 /dev/sdb

Dynamic Free DNS

I used to use everydns.net which was later acquired by DynDNS.  Sadly, once acquired they started migrating users from the free service to their paid subscription service which as far as I can tell all it provides
is the ability to point foobar.com to a dynamic IP, as well as modifying your DNS records (A, CNAME, MX, etc ).

Once alternative is zoneedit.com but my domain seems to be tied into a weird flux having used them in the past that I couldn't use their service as an alternative, so I ended up setting up a custom solution that is probably overkill for most users, but it might be interesting information.

So, I need to point my domain to have 2 name-servers that manages my zone.

I only care about 1 zone really, in my test case I just pointed my 2nd name server to the default my registrar uses.  So, if my primary DNS is down, I'll probably get the standard this page is registered and is owned by foobar.com.

The main issue is that I need to point a registrar to a dynamic IP.  So I setup no-ip service to auto update.  I pointed my Domain registrar to a no-ip address and go figure it accepted it.

So now I can point it to my own IP that can potentially change and dns resolution will go to the proper address.

Now, naturally you need to expose port 53, and you need to run a DNS server of your choice I went for bind.

Now, I can have an CNAME record in my dns that points to say foobar.no-ip.com but all that would give me is a redirect, and I wanted to have my own domain after all.

I need to setup at least one A record that maps to my public IP.  This is easily done, and usually looks something along these lines:

www           IN        A       4.4.4.4

with 4.4.4.4 being your public IP.  The problem that arises is that even though no-ip autoupdates to point outside requests to my DNS, unless I updated bind to point to the right address then there is no point.

My hacky solution was to write this python script that takes a list of bind files to update.

It'll read my dns zone file, search for any IPs not matching my LAN subnet, and update all those records with the current IP address.  Once it finishes, it'll reload bind to make it read the new updated config.

current code is in:  https://github.com/safaci2000/dns_utils though like I said its hacky.

It establishes if an IP is local or not by comparing the first octet.  I need to add some logic that actually checks if the IP is public or not.

So..once this all done I have a domain that points to foobar.no-ip.com for its dns host.

foobar-ip.com points to my machine as long as it's online.  Then I have an hourly updated python script that will update and reload my DNS records.  There is the issue that if my DNS server goes down, nobody
can access my machine.  This normally wouldn't be an issue (for me) since my DNS all points to the local machine, but I am hosting my email with google.  So if my dns server goes down, in theory I could
be losing mail.  I was thinking of just getting a VPS and setting this up a secondary machine, but if I get
to the point of paying $20/mo for a machine to run a DNS zone, then I might as well just pay dyndns my monthly fee.  Then again, a full VPS would be more useful to me and could have some more potential features then just a simple dns updater which is essentially recreated here.

Thoughts, comments?

Making Sense of Eclipse....

Oh Eclipse.. how I love you and how I loath thee.

On linux, I probably think that eclipse is one of the most versetile IDE environments I've ever used.  Pick your language and it probably supports it.  Java, pyhon, c++, cobol as well apparently for those wanting to dust off their mainframe leet skills. 

The main problem I've had with eclipse for ages has been their plugin support.  Yes, they have tons of plugins and most of them work wonderfully.  I start out with the bare eclipse, then I have a need for a python environment.. so I install the plugin, then I need a c++ environment so I install that plugin. 

Then I realize that I have so much clutter in my IDE that I can't find anything.  I also think that if I'm working on a c++ project, I most likely don't want, care or need the python support in my IDE.  Its only purpose is to suck even more of my precious RAM.

I hate having so much clutter.. so I usually end up with 3-4 installs of eclipse which is really silly.

Here's my attempt at fixing this.  (yes, this is super nerdy but it mostly works).

cd $HOME/local; mkdir eclipse
 
extract the latest eclipse into that directory, and add your standard language
neutral customization.
I like using the viPlugin or eclimd and the colorthemeplugin.


Standard install, so extract your eclipse, setup your plugins.. then go to your eclipse install directory (I usually install in $HOME for my own sanity if nothing else)

cd eclipse; git init; git add * 

Also include any dot files in the eclipse directory.

git commit -a -m "initial install"; git tag bare

now, let's setup our java environment.

git checkout master -b java

Get your favorite java plugins.  Maven?  GUI Builder? whatever you like using.

Add everything that's been added to the repo.

git commit -a -m "Adding java stuff"

Now, let's do the python environment.

git checkout master -b python

Install the python plugin and any tools.

Once this is all setup...  you checkout the branch you want to work on different versions of eclipse that are slimmed down. 
You can checkout the bare install at anytime (git checkout bare -b newFork).

In theory it saves you some space, and lets you get somewhat of a handle on your environment.

Well.. that or its a completely random crazy idea that makes no sense to anyone but me.... but sharing anyways.

Thoughts? Comments?

Also, if anyone knows of a way to unload plugins without uninstalling them completely, please let me know.

An interesting but mostly pointless exercise.

So, let me explain my mind frame when I was approaching this.  Some of it is quasi logical, other parts is just me being weird.

For some reason or other.. I wanted to have a few raw block devices.  They're usually created using dd command.

ie.  dd if=/dev/zero of=foobar.raw bs=1M count=2048
bs = block size so the size of foobar.raw = 1m * 2048 = 2048mb = 2 gb.

Now some operations you can perform directly on the file foobar.raw (like formatting it), but since I'm trying to do crytpo + lvm, I need to at least pretend that it's a device.  losetup lets me associate a device name, in my case /dev/loop0 with a file on my file system.

So...  here's my instruction set to get a file to behave like a file system, and allow you to do all sorts of unholy things to it.

**Most of these commands assume root.  Either prepend sudo to all of these, or just become root via su - or your favorite procedure to get god mode.

1.  Create a raw file.

    # dd if=/dev/zero of=foobar.raw bs=1M count=2048

2.  Losetup, to associate file with a device name.

    # losetup /dev/loop0  ./foobar.raw


3.  Cryptosetup, really really not needed, but hey.. why not. 

3a.  Format the device we just created (loop0) to be cryptsetup device.

cryptsetup -y --cipher aes-cbc-essiv:sha256 --key-size 256 luksFormat /dev/loop0

3b.  Lets open the device, and give the encrypted device a name. 

 cryptsetup luksOpen /dev/loop0 cryptoLoop    ## 

4.  Now that we have an encryption layer.. lets's create an LVM, which allows me to 

extend the size of my file system or join multiple files...and all sorts of craziness/flexibility.

Most of these steps are your standard LVM setup, but we'll note them here for consitency.

pvcreate /dev/mapper/cryptoLoop
vgcreate cryptoLvmGrp
lvcreate -l100%VG -n lvmData cryptoLvmGrp

now, in theory you can format the lvm.

mkfs -t ext3 /dev/cryptoLvmGrp/lvmData
mount /dev/cryptoLvmGrp/lvmData loopback

standard usage rules apply, permissions and such.  Once you're done
using it, you need to shut down all the layers in order.

Shutdown:

1.  umount loopback

2.  vgchange -a n 

3.  crypsetup luksClose cryptoLoop

4.  losetup -d /dev/loop0 

So, in retrospect. I can probably just create an xfs/ext file system and expand the file system as needed... but meh.. I kinda like this elaborate setup.

I'm going post some scripts to automate the creation, loading, and unloading of these systems, though some of the paths for LVM and cryptsetup depends on the distro.

Edit:  https://github.com/safaci2000/RawCryptoLVM  Some really dumb scripts that work for me.  

Gentoo is Amazing!!

This is a flame post on Gentoo.  I found it funny mainly because I used Gentoo for the greater
part of Linux experience.  I really think it's an amazing distribution.  That being said.. enjoy the laughs.

http://funroll-loops.info/

HP Mini 210–1076NR + Ubuntu Netbook 10.10

I had a hell of a time getting the broadband working.  This particular devices comes with
Gobi 2000 and I had to do a lot of silly playing around to get it up and running.  It is odd
that the live CD out of the box supports it, but the final installed version didn't.

If you plan doing an install, I would try to be somewhere you can get a Ethernet cable to
plugin so you can get all these updates and get things squared away.

1. if it's not installed already, (apt-get install gobi-loader) get the firmware from the windows partition.
There should be a folder named Qualcomm in your standard Program Files folder.  Which contains the various firmware.  This document explains which folder to get:.  You need to grab the content of the appropriate folder
and copy it to /lib/firmware/gobi

reboot and everything worked fine for me.  If that fails, you may have to load it manually.
    gobi_loader /dev/ttyUSB0 /lib/firmware/gobi

fyi.  If you have the wrong firmware (at least in my case) it just hanged there indefinitely, while when I had the right firmware it worked almost instantly.

Wireless didn't work for me out of the box, but that's the typical broadcomm being "special" thing.  If you do the usual updates it started working for me again due to penguin magic from the ubuntu repos.   I'm sure there's about 800 sources of how to get this working, so I won't go into it.  If you have trouble.. post a comment, and I'll update the article.

Conversion to Linux Test Case.

Part 0 -- Intro

So, a friend of mine inquired about the joys of Linux, and had a genuine curiosity about the operating system.  I gave up trying to push Linux on people in general.  I find it cause me more headaches then its worth, and if people don't want to learn, no matter how amazing and awesome I think Linux and penguins are it still involves people using something different.  I tend to find that the general population, no matter their intellect and abilities are petrified of change.

Part 1 -- Poor Guinea Pig.

My friend is an art student, with an aptitude for computers and technology.  She won't be pulling up the terminal and starting some gcc, and patching the kernel, but she's capable of basic common sense and critical thinking.  She's worked in a computer lab before.... so I'd call her a power win user? somewhere along those lines.

She loves her shiny android, and nook, and just recently discovered that they're Linux based and still marvels at all the tidbits of Linux hidden away in various technologies.

I personally have been using Linux for so long, that I've lost touch with what's easy and isn't, so I thought she'd be a good use case.

Part 2 -- Target machine 

She just recently purchase a Netbook from verizon, with Windows 7 start edition.

Windows 7 Starter issues:
1.  limited to 3 concurrent process at a time.
2. Can't change the wallpaper.

Requirements:
Office of some form.  (OpenOffice/google docs worked)
IPod (Banshee /Rhythmbox seemed to work )

I had her look at Linux Mint, Ubuntu Netbook edition, and Fedora...and she liked the Unitiy look.

I tried walking her through an install, and that failed miserably once the idea of partitioning, mount points and all that came up, so instead decided to do it in person.

Netbook:  HP Mini 210–1076NR

Live CD issues.  Wireless didn't work.   Possible mouse issues.

I partitioned the machine proper using gparted and set it up to dual boot windows 7 and Ubuntu.  I went through the install process, everything seemed to work fine once it was done with a few exceptions.

Wireless was still broken, but that's okay.  We had the Verizon broadband, went online.. had to do some manual tasks to get the firmware download and get the silly wifi working.  First time using unity, so it didn't click right away, but the mouse won't right click.  (xorg/touchpad config probably).

I got her wifi updated, then had her do the updates she was prompted for.  The updates broke her Verizon broadband.and I tracked it down to a bug:

** That's a big no-no.  Ubuntu is supposed to be super user friendly... once you make a release, you can't break drivers by doing an update.  Especially when the fix is patch the kernel and recompile... or go manually find the firmware for your card.

This worked out of the box... it should not be this difficult for an end user to get things fixed.  If the target audience is ease of use, intricate command line tools, patching code, reloading kernel module.. is not a proper way of handling things.

3.  Final thoughts.. so far.

So, I seemed more annoyed with Ubuntu behavior then she was.  She seems very happy with speed/performance over windows 7.

Enjoys all the random apps that free and available to use.  I have to fix her mouse still, but aside from Verizon and a bit of rocky start everything else works, but it involved more techie experience then it should have.

Many of these things should just work out of the box.


PS.  I need to track down the exact bug number, I'll link it in an update.