Post Reply 
 
Thread Rating:
  • 2 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Cross-compiling for DroboFS series: Getting the VM set up
09-08-2010, 11:20 AM (This post was last modified: 03-16-2011 04:55 PM by ricardo.)
Post: #1
Cross-compiling for DroboFS series: Getting the VM set up
EDIT: I have updated this HOWTO and simplified the procedure a lot! There is no more need for a chroot inside the VM, just a VM with the latest version of Ubuntu and the toolchain from CodeSourcery. Updated version can be found here: http://www.droboports.com/setting-up-a-vm



Alright, this one is the first post in a series about how to cross-compile some (in my opinion) essential stuff to make the most out of those amazing machines.

Let's start with the compiling environment. First of all, credit where credit is due: most of the hard work here comes from http://drobo.jhah.net/guides/cross_compile_setup, and I just want to point out some things that will make the procedure a bit more streamlined for others treading this path.

Step 1: Get a Virtual Machine environment.

Why use a VM? For several reasons, but primarily because:
  1. Allows you to keep the compiling environment lean
  2. Makes it easier to prevent unwanted system upgrades (e.g. breaking compatibilities)
  3. Allows you to easily move the environment from one machine to another
  4. Prevents any possibility of messing up your hosting machines
  5. The performance impact of compiling in a virtual machine is minimal (if properly configured, i.e., use ssh instead of the VM window)

I am using VirtualBox (from Sun/Oracle - http://www.virtualbox.org/wiki/Downloads). The instructions here do not necessarily require VirtualBox, and in fact any virtualization software that allows inbound network connections will do fine.

Step 2: Create a Virtual Machine.

The configuration of the VirtualBox VM I am using is:
  • System: Acceleration: Enable VT-x/AMD-V, Enable Nested Paging
  • RAM: 512 MB
  • Video Memory: 12MB
  • Storage: 4GB
  • Network: At least one bridged or host-only adaptor (to allow inbound ssh connections)

Step 3: Install an OS.

I went for a recent, small version of Ubuntu. The one I picked is the Ubuntu Minimal 10.04, that can be found here: https://help.ubuntu.com/community/Instal...https://help.ubuntu.com/community/Installation

I chose the 32-bit version for no other reason that it would make it easier to run the VM on older computers, and I think it reduces the chances of problems later on with chroot.

While installing, pick only the SSH server from the Ubuntu install menu. All the rest is unnecessary. This is the reason we need a "externally accessible" network interface. Instead of using the VM's window (which is usually terribly slow due to video emulation), we will ssh into the VM and run from there. The speed difference is worth it.

Step 4: Create a chroot for Drobo compiling.

Here is where the magic begins. A very nice description of what needs to be done is presented here: http://drobo.jhah.net/guides/cross_compile_setup

A couple of points that I need to emphasize, though. Make sure you got the right toolchain. For some reason I downloaded the non-gnu version and was banging my head for a couple of hours to understand why the heck nothing would compile...

The toolchain needed can be found here: http://www.codesourcery.com/sgpp/lite/arm/
More especifically, the one I used is: http://www.codesourcery.com/public/gnu_t...http://www.codesourcery.com/public/gnu_toolchain/arm-none-linux-gnueabi/arm-2007q1-21-arm-none-linux-gnueabi-i686-pc-linux-g

One last thing: I suggest creating a small shell script with the lines
Code:
mount -o bind /proc /var/chroot/drobofs/proc
schroot -c drobofs -d /root
...to make sure that they are called every time you enter the chroot.

Step 5: Make yourself cozy chroot environment.

Alright, if everything was done properly, you have a nice empty chroot. Almost too empty. Let's make it more amenable.

First, let's get the essentials:
Code:
apt-get install automake autoconf libtool autotools-dev m4

Now some amenities:
Code:
apt-get install nano wget smbfs

I'm a nano guy - feel free to use any editor you want. You might be wondering why have smbfs in there as well. It will be used to transfer what we compiled from the VM to the DroboFS more easily (see below).

Now, create a script file containing this:
Code:
export PATH=/usr/local/arm-2007q1/bin:$PATH
export CFLAGS="-I/usr/arm/include -march=armv5te"
export CPPFLAGS=${CFLAGS}
export CXXFLAGS=${CFLAGS}
export LDFLAGS="-L/usr/arm/lib"
export CC=arm-none-linux-gnueabi-gcc
export CXX=arm-none-linux-gnueabi-g++
export AR=arm-none-linux-gnueabi-ar
export RANLIB=arm-none-linux-gnueabi-ranlib
I called it "crosscompile.sh" on my home folder and placed it on the end of my .bashrc to make sure that it was sourced every login, like this:
Code:
. ~/crosscompile.sh

Finally, create a folder where all the downloaded source code will be placed, the folder where all cross-compiled binaries will be placed, and the folder where a smbfs mount point will allow the transfer to the DroboFS:
Code:
mkdir -p /root/code
mkdir -p /usr/arm
mkdir -p /mnt/DroboApps

Step 6: Configure the smbfs mount point.

Edit the file /etc/fstab within the chroot environment and add this line to it:
Code:
# other file systems here
//drobofs/DroboApps    /mnt/DroboApps smbfs domain=<your_domain>,user=<your_drobo_username>,password=<your_drobo_password>

Now you can mount it by doing:
Code:
sudo mount /mnt/DroboApps

A quick "ls" in there should show you the content of the DroboApps share.

Since we have this working, make sure that you have write access to the share by creating a folder that will contain all the compiled stuff later on:
Code:
mkdir /mnt/DroboApps/arm

Step 7: Pat yourself on the back.

That is it. From this point on you are ready to get to the hard part of the job, which is download the code, configure, make and transfer it to the DroboFS.

Which reminds me...

Step 8: How to transfer the compiled goodies from the VM to the DroboFS.

Once you get some stuff compiled, (hopefully) all of it will be placed within the /usr/arm folder. In other words, executables will be placed under /usr/arm/bin, libraries will be under /usr/arm/lib, and so on.

The transfer to the DroboFS is made in four steps:
  1. package all the files under /usr/arm in one big archive:
    Code:
    cd /usr/arm
    tar cjfv ../arm.tar.bz2 .
  2. transfer arm.tar.bz2 to the DroboFS using the smbfs mount created in step 6:
    Code:
    cp ../arm.tar.bz2 /mnt/DroboApps/arm/ (edit: source path fixed)
  3. unpackage the archive. SSH to the DroboFS* and type:
    Code:
    cd /mnt/DroboFS/Shares/DroboApps/arm
    tar jxvf arm.tar.bz2

*) I know that you could just copy all the files directly using the mount, however it has been my experience that it is much faster to copy one big file than copy hundreds of small files over SMB. YMMV.

Step 9: How to use the goodies on the DroboFS.

Just copying it over is not enough. There are a couple of adjustments that need to be done on the DroboFS to make it all work.

WARNING: from this point on we are making changes to the DroboFS root filesystem. If you are not sure of what you are doing, then please do not proceed. I decline all responsibility in case of a major screw up, and I hope that a system reset may salvage it. Seriously, don't do it unless you know how to undo it.

First of all, ssh to the DroboFS and create a symlink from /usr/arm to /mnt/DroboFS/Shares/DroboApps/arm:
Code:
cd /usr
sudo ln -s /mnt/DroboFS/Shares/DroboApps/arm arm

Now, edit whatever shell configuration you are using to add those two lines (in my case, that was ~/.bashrc):
Code:
export PATH=/usr/arm/bin:$PATH
export LD_LIBRARY_PATH=/usr/arm/lib:/lib

After the next login, all the compiled stuff in /mnt/DroboFS/Shares/DroboApps/arm should be available just like any other application on the system.

Step 10: Use your DroboFS to backup the VM.

Now that you have a fancy VM that allows you to compile software for your DroboFS, what better way to make this self-referencial than to backup a pristine copy of the VM in the DroboFS? This way, if you ever screw up the VM bad enough for it not to work anymore you'll always be able to restore a fresh copy of it without having to go through all of this again.

Final words:

I know that this seems like a painful and arduous task, but it is not that hard actually. Overall, most of the time is taken by waiting the VM to install the OS and making sure you copied the information correctly. A total beginner should take about a couple of hours to do this. Someone with VM and OS setup experience should be done in 30 minutes on a good machine.

Finally, I'm not sure that it makes sense for me to provide the disk file for the virtual machine for download. The image is 4GB, where half of it is empty space and most of the non-empty space is just part of the OS install. In the time that it'll take to download it and (assuming you are using the same VM software as I am) configure it to run on another system, one can easily do the whole procedure from scratch, probably even faster.

Comments, suggestions and feedback are welcome.

- Changelog:
- Oct 4th: fixed source path on step 8.2 - thanks ZenCocoon for the feedback.
Find all posts by this user
Quote this message in a reply
10-03-2010, 08:46 AM (This post was last modified: 10-03-2010 09:16 AM by ZenCocoon.)
Post: #2
RE: Cross-compiling for DroboFS series: Getting the VM set up
Hi Ricardo,

Thanks a lot for your excellent tutorial, really helpful.

Little question, once on the DroboFS, you are updating the PATH and LD_LIBRARY_PATH using .bashrc
How have you done to use bash in first place? On my side, and I guess by default, it uses ash and doesn't know how to deal with it to be honest.

Thanks again for sharing your progress on the DroboFS,
--
Sébastien Grosjean - ZenCocoon
Tiny fix step 8.2 I guess you mean :

Code:
cp ../arm.tar.bz2 /mnt/DroboApps/arm/

Thanks again for your tutorials
Find all posts by this user
Quote this message in a reply
10-03-2010, 09:18 AM
Post: #3
RE: Cross-compiling for DroboFS series: Getting the VM set up
Hmm, I might have compiled it for the DroboFS and forgot about it.

Anyways, I have a /bin/bash:
Code:
-rwxr-xr-x    1 root     root      1488261 Jul 31 20:38 /bin/bash

Once you have that file in place, you need to edit /etc/passwd to make it from this:
Code:
root:<password hash here>:0:0:root:/mnt/DroboFS:/bin/ash
to this:
Code:
root:<password hash here>:0:0:root:/mnt/DroboFS:/bin/bash

In other words: do not change anything but the last item on that line.

Once that is edited, you can give it a try to see if t is working. If it is not working, i.e., you can't login anymore, just reset the DroboFS. The file /etc/passwd will be reset to its original state.

By the way, if you can't login this means that either you changed other stuff than the last item on that line, or that the file /bin/bash does not exist or is broken.

After that, if you are absolutely sure all is correct, and I mean really, really, 100% absolutely sure that everything is right (maybe you should sleep on it?), then you can change the file /etc/.passwd to match the change so that it will survive reboots.

You know what, on second thought, don't change /etc/.passwd, otherwise Jennifer will get mad at me. Smile

Cheers,
Ricardo
Find all posts by this user
Quote this message in a reply
10-03-2010, 09:23 AM
Post: #4
RE: Cross-compiling for DroboFS series: Getting the VM set up
lol, got the point with /etc/.password , feels good to have it persistent over reset but understand that if broken, there's no more chance to come back after that ;-)

However, will certainly feels stupid but how did you got bash installed on the drobofs in first place?

Did you compiled it as well on the VM?

Thanks for your quick reply,
--
Sébastien Grosjean - ZenCocoon
Find all posts by this user
Quote this message in a reply
10-03-2010, 10:11 AM
Post: #5
RE: Cross-compiling for DroboFS series: Getting the VM set up
(10-03-2010 09:23 AM)ZenCocoon Wrote:  However, will certainly feels stupid but how did you got bash installed on the drobofs in first place?

Did you compiled it as well on the VM?

Hmm, if you don't have the executable in there already, then I probably compiled it as well.

Funny thing is since I did not have it on my compilation notes I was pretty sure that it was already there.

I'll check my VM, and post a compilation guide as soon as I can.

Cheers,
Ricardo
Find all posts by this user
Quote this message in a reply
10-03-2010, 11:25 AM
Post: #6
RE: Cross-compiling for DroboFS series: Getting the VM set up
sounds great.

May I use your tutorial template to post my successful compilation stories ?

Cheers,
--
Sébastien Grosjean - ZenCocoon
Find all posts by this user
Quote this message in a reply
10-03-2010, 02:04 PM
Post: #7
RE: Cross-compiling for DroboFS series: Getting the VM set up
(10-03-2010 11:25 AM)ZenCocoon Wrote:  May I use your tutorial template to post my successful compilation stories ?

I would be honored. Smile
Find all posts by this user
Quote this message in a reply
10-03-2010, 02:55 PM (This post was last modified: 10-03-2010 03:02 PM by ZenCocoon.)
Post: #8
RE: Cross-compiling for DroboFS series: Getting the VM set up
By the way, don't forget the tiny fix step 8.2 I guess you mean :

Code:
cp ../arm.tar.bz2 /mnt/DroboApps/arm/

Thanks again for your tutorials




(10-03-2010 02:04 PM)ricardo Wrote:  
(10-03-2010 11:25 AM)ZenCocoon Wrote:  May I use your tutorial template to post my successful compilation stories ?

I would be honored. Smile

Thanks a lot, just finished to compile ruby 1.8.7, was a big pain but it's now running fine. I'll try to put the tutorial up shortly.

cheers,
--
Sébastien Grosjean - ZenCocoon
Find all posts by this user
Quote this message in a reply
10-28-2010, 11:49 AM
Post: #9
RE: Cross-compiling for DroboFS series: Getting the VM set up
When attempting to step: # create /etc/schroot/chroot.d/drobofs, the DroboFS returns "Access Denied."

Permissions issue? chmod in order? I've followed the steps to that point meticulously, but can't seem to get past this.
Find all posts by this user
Quote this message in a reply
10-28-2010, 01:29 PM
Post: #10
RE: Cross-compiling for DroboFS series: Getting the VM set up
(10-28-2010 11:49 AM)FlyBoyOz Wrote:  When attempting to step: # create /etc/schroot/chroot.d/drobofs, the DroboFS returns "Access Denied."

Permissions issue? chmod in order? I've followed the steps to that point meticulously, but can't seem to get past this.

I think you are confusing the Virtual Machine with the DroboFS. All the steps on that page are supposed to be done inside the VM, not on the DroboFS.

Also, keep in mind that 'cat' call should be a 'sudo cat', since you are trying to create a file in a root-owned directory.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: