The proper way to do this is to run System Settings -> Display and drag the black bar at the top of your primary monitor to another monitor. I find the Display interface really difficult to use and unresponsive.
So, as much fun as it is to try to get that UI to work, this is my work around using the CLI.
list all your monitors:
xrandr
This will list the IDs of all your current monitors. To set a different monitor as the default type something along these lines:
xrandr --output HDMI3 --primary
where HDMI3 is the ID of the monitor you want set as your primary monitor.
Wednesday, November 21, 2012
Thursday, November 15, 2012
Converting from Raid 1 to Raid 5
This assumes you have a functional Raid 1 and wish to convert it to a Raid 5.
Disclaimer: At some point during this process I realized that I had a bad mother board. The reason my /dev/sdd1 failed wasn't the drive, but the bus on the board. That being said, this is the unverified procedure.
I'm running on Ubuntu 12.10 but you should be able to do this on any modern Linux distribution.
1. Stop all mdadm related tasks.
sudo umount /media/data #of whatever mountpoint is appropriate.
sudo /etc/init.d/mdadm stop
sudo mdadm --stop /dev/md0
2. Change the raid layout
This part is kind of scary, and I wouldn't advice mounting the raid at this point. I especially didn't like the fact that it looks like it's overwriting my raid.. that made me nerveous, but it's essentially restrucuting how data is stored, and putting a raid 5 mapping on 2 drives. ie. creating a degraded raid 5.
#update this as appropriate.
mdadm --create /dev/md0 --level=5 --raid-devices=2 /dev/sda1 /dev/sdd1
WARNING: wait for this step to complete.. look at /prod/mdstats and wait for it to finish before proceeding.
3. Add a 3rd drive.
#in my case I started with /dev/sdd1, added /dev/sda1 to create a raid 1, and then adding /dev/sdb1 as the final device. You don't have to follow my convention. That made sense for my use case, since my dead drive was /dev/sdd you can simply start with /dev/sda1 and go alphabetical.
mdadm --add /dev/md0 /dev/sdb1
mdadm --grow /dev/md0 --raid-devices=3
This part took around 15 hours for me.
At this point, I'd be okay with mounting my raid partition. Again, it's safer not to... but.. it's won't break the process if you do.
4. Expand File System.
At this point what we have is the equivalent of have a large hard drive, but a smaller partition on it. We need to grow the local file system.
I'm covering 3 use cases.
a. LVM
I've have had lvm on my raid before. Actually, I used to have raid + lukefs encryption + lvm. Too many layers though the performance isn't as bad as you might expect.
TODO: I have to look this up... I'll update this eventually.
b. XFS
xfs is a bit odd, you need to have the file system mounted in order to grow it.
xfs_repair -n /dev/md0 #just to be safe.
mount /dev/md0 /media/data
xfs_growfs /media/data
c. EXT3/4
e2fsck -f /dev/md0 #check file system
resize2fs /dev/md0 #grow file system
5. Update fstab
This should not need any changes, but just in case:
/dev/md0 /media/data xfs defaults
6. Update mdadm.conf
sudo su -
mdadm --detail --scan >> /etc/mdadm/mdadm.conf
##edit the file and remove the next to last line. ie. The command above appends the new mdadm config to your config file. So remove the previous raid 1 line. There should be a single line defining md0 which looks something like this:
ARRAY /dev/md/md0 metadata=1.2 UUID=0ec3c5aa:5cee600b:ef1e8f7d:09b20cc8
This is the line I removed:
#ARRAY /dev/md/md0 metadata=0.90 UUID=bf8a2737:554e654c:c2eab133:b01f9710
In other words, assuming you only have 1 raid setup, your mdadm.conf should only have a single ARRAY configured.
References:
Disclaimer: At some point during this process I realized that I had a bad mother board. The reason my /dev/sdd1 failed wasn't the drive, but the bus on the board. That being said, this is the unverified procedure.
I'm running on Ubuntu 12.10 but you should be able to do this on any modern Linux distribution.
1. Stop all mdadm related tasks.
sudo umount /media/data #of whatever mountpoint is appropriate.
sudo /etc/init.d/mdadm stop
sudo mdadm --stop /dev/md0
2. Change the raid layout
This part is kind of scary, and I wouldn't advice mounting the raid at this point. I especially didn't like the fact that it looks like it's overwriting my raid.. that made me nerveous, but it's essentially restrucuting how data is stored, and putting a raid 5 mapping on 2 drives. ie. creating a degraded raid 5.
#update this as appropriate.
mdadm --create /dev/md0 --level=5 --raid-devices=2 /dev/sda1 /dev/sdd1
WARNING: wait for this step to complete.. look at /prod/mdstats and wait for it to finish before proceeding.
3. Add a 3rd drive.
#in my case I started with /dev/sdd1, added /dev/sda1 to create a raid 1, and then adding /dev/sdb1 as the final device. You don't have to follow my convention. That made sense for my use case, since my dead drive was /dev/sdd you can simply start with /dev/sda1 and go alphabetical.
mdadm --add /dev/md0 /dev/sdb1
mdadm --grow /dev/md0 --raid-devices=3
This part took around 15 hours for me.
At this point, I'd be okay with mounting my raid partition. Again, it's safer not to... but.. it's won't break the process if you do.
4. Expand File System.
At this point what we have is the equivalent of have a large hard drive, but a smaller partition on it. We need to grow the local file system.
I'm covering 3 use cases.
a. LVM
I've have had lvm on my raid before. Actually, I used to have raid + lukefs encryption + lvm. Too many layers though the performance isn't as bad as you might expect.
TODO: I have to look this up... I'll update this eventually.
b. XFS
xfs is a bit odd, you need to have the file system mounted in order to grow it.
xfs_repair -n /dev/md0 #just to be safe.
mount /dev/md0 /media/data
xfs_growfs /media/data
c. EXT3/4
e2fsck -f /dev/md0 #check file system
resize2fs /dev/md0 #grow file system
5. Update fstab
This should not need any changes, but just in case:
/dev/md0 /media/data xfs defaults
6. Update mdadm.conf
sudo su -
mdadm --detail --scan >> /etc/mdadm/mdadm.conf
##edit the file and remove the next to last line. ie. The command above appends the new mdadm config to your config file. So remove the previous raid 1 line. There should be a single line defining md0 which looks something like this:
ARRAY /dev/md/md0 metadata=1.2 UUID=0ec3c5aa:5cee600b:ef1e8f7d:09b20cc8
This is the line I removed:
#ARRAY /dev/md/md0 metadata=0.90 UUID=bf8a2737:554e654c:c2eab133:b01f9710
In other words, assuming you only have 1 raid setup, your mdadm.conf should only have a single ARRAY configured.
References:
Tuesday, November 13, 2012
Raid Upgrade Trick
Premise: Here's my situation. I have a raid 5, with 3 1 TB drives, which gives me about 1.8 give or take of storage. I had one drive go bad on it, and I found a great deal on amazon, so I decided to upgrade to a new raid 5 configuration with 3x 2 TB drives.
To recap.
Current setup:
Let's add another drive to the array.
mdadm --grow /dev/md0 -n 2
mdadm --manage /dev/md1 --add /dev/sda1
If the above command works, then if you issue a
cat /proc/mdstat
You should see data being synced.
This gives me a new functional raid 1. We're not done. The next step is to convert a raid 1 to raid 5, and since I still have 2 functional 1 TB drives where I can reconstruct my old raid with, I also have a backup of the data.. so if I lose everything.. I just lose another day or so to copy data back.
Side note: ETA to copy 1.1 TB of data was about +8 hours. I started around 6 pm and it was still going at 2 am.
ETA to do the sync above is estimated at 4-5 hours. Though thankfully I can leave my machine up and running and use it while it's synching.
(I forgot to mention, if you want to access your data while it's synching.. you can mount the drive:
sudo mount /dev/md127 /media/data
naturally it's recommended not to use your computer while doing that.. but meh... next step will take even longer... (Raid 1 to Raid 5 conversion). I was actually playing a game while I was copying the data (rsync). Though I did re-run the command once more once everything was done to make sure it got everything.
Look forward to the next article while will talk about the raid conversions.
References:
1. Create A Degraded Raid 1 Array
To recap.
Current setup:
- 3 drives
- 1 TB of space for each
- Raid 5
- total raid capacity: around 1.8
- space used: 1.3 TB
New Setup:
- 3 drives
- 2 TB of space for each
- Raid 5
- total raid capacity once completed: around 3.8 TB
Problem:
1. I don't have enough ports on my machine to plugin all of the drives for my 2 raids. Ideal would require 6 sata ports for the 2 raids and a 7th one for my OS drive. Even if I futz with things, and allow for one drives to be missing from each. (ie. run a degraded raid), I would still need 5 ports, which I don't have.
Solution:
Step 1. Copy data
Note: This can be done a bit differently...but I think my approach will save you time in the long run.
I replaced the dead 1 TB drive with one of the new drives. The computer starts up, It brings up the old raid /dev/md0 in a degraded state per usual, and I now have 1 new drive to play with that I know is good.
Using the new drive, I'm creating a new degraded raid 1 device.
mdadm --create /dev/md1 -l raid1 -f -n 1 /dev/sdd1
mkfs -t xfs (or whatever you prefer) /dev/md1
sudo mount /dev/md1 /media/newRaid
rsync -avP /media/oldRaid/ /media/newRaid/ ## be careful the / matter and change rsync's behavior.
So at this point, we have 2 raids up and running, and I'm copying data from the old raid to the new one.
Step 2. Replace Drives
At this point, shut down the machine, remove your remaining drives from the old raid, and replace them with the new one. comment out your /etc/fstab if you're mounting your raid by default.
It will probably create some random /dev/md* depending on your OS and intelligent it is. Ubuntu re-created the raid array as /dev/md127, which I'll use for now. we can easily rename it to md0 once everything is done.
It will probably create some random /dev/md* depending on your OS and intelligent it is. Ubuntu re-created the raid array as /dev/md127, which I'll use for now. we can easily rename it to md0 once everything is done.
Let's add another drive to the array.
mdadm --grow /dev/md0 -n 2
mdadm --manage /dev/md1 --add /dev/sda1
If the above command works, then if you issue a
cat /proc/mdstat
You should see data being synced.
VERY IMPORTANT... wait for the sync to finish before doing anything else at this point.
This gives me a new functional raid 1. We're not done. The next step is to convert a raid 1 to raid 5, and since I still have 2 functional 1 TB drives where I can reconstruct my old raid with, I also have a backup of the data.. so if I lose everything.. I just lose another day or so to copy data back.
Side note: ETA to copy 1.1 TB of data was about +8 hours. I started around 6 pm and it was still going at 2 am.
ETA to do the sync above is estimated at 4-5 hours. Though thankfully I can leave my machine up and running and use it while it's synching.
(I forgot to mention, if you want to access your data while it's synching.. you can mount the drive:
sudo mount /dev/md127 /media/data
naturally it's recommended not to use your computer while doing that.. but meh... next step will take even longer... (Raid 1 to Raid 5 conversion). I was actually playing a game while I was copying the data (rsync). Though I did re-run the command once more once everything was done to make sure it got everything.
Look forward to the next article while will talk about the raid conversions.
References:
1. Create A Degraded Raid 1 Array
Tuesday, November 15, 2011
Wednesday, September 14, 2011
Migrating from Mailman to Google Groups
Migration Process
I've recently migrated a mailing list from a server running postfix + mailman to a google group with the idea
of ease of management and removing the need of having a physical server.
Someone asked me to comment on my thoughts on the migration via email, so I'm mostly sharing the same info for anyone who might have an interest.
-----
I retrieved the full listing of all emails from mailman via the CLI
> ./list_members <listname> > members.txt
Then login to the google groups interface paste the file in and invite all your members
to join the new mailing list. I made the list moderated and set an autoreply to all incoming messages telling the members to join the google group.
I waited about a week or two for people to join and such... then I changed the DNS entries and moved the mail server to a google hosted domain. I recreated a user with the previous list name...
so if my mailing list name was javaFanBoys@foobar.com I created a new user with that same
address and created a vacation message instructing the user that the old email list has moved over.
In my case I had an announce list for events that forwarded all its emails to the more general list.
I haven't figured out how to make it broadcast to all users though there is an archiver email that takes whatever email it receives and treats it as if it was a regular email to the list in all regards except the fact
that it doesn't send it out as an email to the user base. (ie visible via web view but not via email)
Under your groups advanced setting you'll have an email along these lines: listname+garchive-###@googlegroups.com. So I created a new account again named bot, which is subscribed to both lists, and any emails received from my -announce list gets forward to the email address listed above.
Motivation
The reason I decided to move over was for:
I don't like having to rely on a single person who has the master password to make any major changes. The delegation of rights via google is a lot easier. You can setup email distribution groups like info@domain or help@domain to allow everyone to be involved and informed.
Problems
Here are the current limitations of google groups.
I've recently migrated a mailing list from a server running postfix + mailman to a google group with the idea
of ease of management and removing the need of having a physical server.
Someone asked me to comment on my thoughts on the migration via email, so I'm mostly sharing the same info for anyone who might have an interest.
-----
I retrieved the full listing of all emails from mailman via the CLI
> ./list_members <listname> > members.txt
Then login to the google groups interface paste the file in and invite all your members
to join the new mailing list. I made the list moderated and set an autoreply to all incoming messages telling the members to join the google group.
I waited about a week or two for people to join and such... then I changed the DNS entries and moved the mail server to a google hosted domain. I recreated a user with the previous list name...
so if my mailing list name was javaFanBoys@foobar.com I created a new user with that same
address and created a vacation message instructing the user that the old email list has moved over.
In my case I had an announce list for events that forwarded all its emails to the more general list.
I haven't figured out how to make it broadcast to all users though there is an archiver email that takes whatever email it receives and treats it as if it was a regular email to the list in all regards except the fact
that it doesn't send it out as an email to the user base. (ie visible via web view but not via email)
Under your groups advanced setting you'll have an email along these lines: listname+garchive-###@googlegroups.com. So I created a new account again named bot, which is subscribed to both lists, and any emails received from my -announce list gets forward to the email address listed above.
Motivation
The reason I decided to move over was for:
- easier management and delegating admins/moderators and such
- removing required infrastructure
- migrate the main website to sites.google.com
- consolidated callendar of events of various associated groups in the area
I don't like having to rely on a single person who has the master password to make any major changes. The delegation of rights via google is a lot easier. You can setup email distribution groups like info@domain or help@domain to allow everyone to be involved and informed.
Problems
Here are the current limitations of google groups.
- There is no easy way of importing old posts short of re-emailing all the history to the list.
- It's difficult to forward emails from one group to another and have it be broadcasted to users.
- sites.google.com is great for simple content, but if you want to edit html or do anything remotely fancy it is very limiting.
Thursday, September 8, 2011
Create a Windows Boot Disk from Linux
I tend to not have any Windows machines around unless I need them, and I'm getting tired of the unlabeled CDs floating all around me. I decided to see if I could figure out a way on how to make a bootable windows USB thumb drive from Linux.
initial solution can be found here
initial solution can be found here
1. format your usb thumb drive to NTFS.
sudo mkfs -t ntfs /dev/sdb1
2. make the partition bootable:
#fdisk /dev/sdb
a (toggle bootable flag)
1 (select partition I'm presuming you only have a single partition )
w (write changes)
3. mount your cdrom (which should happen automatically ) or the ISO.
# mount -o loop win7.iso win_disk
4. mount the usb drive.
# mount /dev/sdb1 /mnt/usb
5. copy data over.
#rsync -avP /mnt/cdrom/ /mnt/usb/
6. Download and build ms-sys which you can get from here:
7. Make USB thumb drive bootable
# ms-sys -7 /dev/sdb
Thursday, June 9, 2011
Dynamic Free DNS
I used to use everydns.net which was later acquired by DynDNS. Sadly, once acquired they started migrating users from the free service to their paid subscription service which as far as I can tell all it provides
is the ability to point foobar.com to a dynamic IP, as well as modifying your DNS records (A, CNAME, MX, etc ).
Once alternative is zoneedit.com but my domain seems to be tied into a weird flux having used them in the past that I couldn't use their service as an alternative, so I ended up setting up a custom solution that is probably overkill for most users, but it might be interesting information.
So, I need to point my domain to have 2 name-servers that manages my zone.
I only care about 1 zone really, in my test case I just pointed my 2nd name server to the default my registrar uses. So, if my primary DNS is down, I'll probably get the standard this page is registered and is owned by foobar.com.
The main issue is that I need to point a registrar to a dynamic IP. So I setup no-ip service to auto update. I pointed my Domain registrar to a no-ip address and go figure it accepted it.
So now I can point it to my own IP that can potentially change and dns resolution will go to the proper address.
Now, naturally you need to expose port 53, and you need to run a DNS server of your choice I went for bind.
Now, I can have an CNAME record in my dns that points to say foobar.no-ip.com but all that would give me is a redirect, and I wanted to have my own domain after all.
I need to setup at least one A record that maps to my public IP. This is easily done, and usually looks something along these lines:
www IN A 4.4.4.4
with 4.4.4.4 being your public IP. The problem that arises is that even though no-ip autoupdates to point outside requests to my DNS, unless I updated bind to point to the right address then there is no point.
My hacky solution was to write this python script that takes a list of bind files to update.
It'll read my dns zone file, search for any IPs not matching my LAN subnet, and update all those records with the current IP address. Once it finishes, it'll reload bind to make it read the new updated config.
current code is in: https://github.com/safaci2000/dns_utils though like I said its hacky.
It establishes if an IP is local or not by comparing the first octet. I need to add some logic that actually checks if the IP is public or not.
So..once this all done I have a domain that points to foobar.no-ip.com for its dns host.
foobar-ip.com points to my machine as long as it's online. Then I have an hourly updated python script that will update and reload my DNS records. There is the issue that if my DNS server goes down, nobody
can access my machine. This normally wouldn't be an issue (for me) since my DNS all points to the local machine, but I am hosting my email with google. So if my dns server goes down, in theory I could
be losing mail. I was thinking of just getting a VPS and setting this up a secondary machine, but if I get
to the point of paying $20/mo for a machine to run a DNS zone, then I might as well just pay dyndns my monthly fee. Then again, a full VPS would be more useful to me and could have some more potential features then just a simple dns updater which is essentially recreated here.
Thoughts, comments?
is the ability to point foobar.com to a dynamic IP, as well as modifying your DNS records (A, CNAME, MX, etc ).
Once alternative is zoneedit.com but my domain seems to be tied into a weird flux having used them in the past that I couldn't use their service as an alternative, so I ended up setting up a custom solution that is probably overkill for most users, but it might be interesting information.
So, I need to point my domain to have 2 name-servers that manages my zone.
I only care about 1 zone really, in my test case I just pointed my 2nd name server to the default my registrar uses. So, if my primary DNS is down, I'll probably get the standard this page is registered and is owned by foobar.com.
The main issue is that I need to point a registrar to a dynamic IP. So I setup no-ip service to auto update. I pointed my Domain registrar to a no-ip address and go figure it accepted it.
So now I can point it to my own IP that can potentially change and dns resolution will go to the proper address.
Now, naturally you need to expose port 53, and you need to run a DNS server of your choice I went for bind.
Now, I can have an CNAME record in my dns that points to say foobar.no-ip.com but all that would give me is a redirect, and I wanted to have my own domain after all.
I need to setup at least one A record that maps to my public IP. This is easily done, and usually looks something along these lines:
www IN A 4.4.4.4
with 4.4.4.4 being your public IP. The problem that arises is that even though no-ip autoupdates to point outside requests to my DNS, unless I updated bind to point to the right address then there is no point.
My hacky solution was to write this python script that takes a list of bind files to update.
It'll read my dns zone file, search for any IPs not matching my LAN subnet, and update all those records with the current IP address. Once it finishes, it'll reload bind to make it read the new updated config.
current code is in: https://github.com/safaci2000/dns_utils though like I said its hacky.
It establishes if an IP is local or not by comparing the first octet. I need to add some logic that actually checks if the IP is public or not.
So..once this all done I have a domain that points to foobar.no-ip.com for its dns host.
foobar-ip.com points to my machine as long as it's online. Then I have an hourly updated python script that will update and reload my DNS records. There is the issue that if my DNS server goes down, nobody
can access my machine. This normally wouldn't be an issue (for me) since my DNS all points to the local machine, but I am hosting my email with google. So if my dns server goes down, in theory I could
be losing mail. I was thinking of just getting a VPS and setting this up a secondary machine, but if I get
to the point of paying $20/mo for a machine to run a DNS zone, then I might as well just pay dyndns my monthly fee. Then again, a full VPS would be more useful to me and could have some more potential features then just a simple dns updater which is essentially recreated here.
Thoughts, comments?
Subscribe to:
Posts (Atom)