Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
global symlinks to executables
07-20-2016, 09:29 AM
Post: #1
global symlinks to executables
I noticed that almost all of the generic command-line utility ports (htop, nano, git, etc.) do not make global symlinks to their executables in /usr/bin, which hampers their usefulness and, I'd say, defies one's expectation. (Of course, one could add each and every installed app's 'bin' folder to the user's $PATH, but that's not fun or maintainable.)

The SDK PDF (page 12) mentions symlinks in /usr/bin being completely reasonable for the cases I'm talking about:

Quote:The DroboApp wants to symlink well known locations to its base folder. For example, the perl5 DroboApp symlinks /usr/bin/perl (which does not exist in the current firmware) to /mnt/DroboFS/Shares/DroboApps/perl/bin/perl, to facilitate the execution of Perl scripts.

Two examples of DroboApps I've seen that do symlink their executables is bash (to /bin/bash) and python (to /usr/bin/python).

I'd be happy to submit PRs to various port repos that add the symlinks (and clean them up during uninstall), but I didn't want to go through the trouble if it would be violating some convention or rationale.
Find all posts by this user
Quote this message in a reply
07-20-2016, 07:38 PM
Post: #2
RE: global symlinks to executables
As the author of the python, perl, and bash DroboApps, please let me explain the rationale behind it.

I have used these apps as a testing ground for that level of integration. It turns out that this kind of integration is rarely, if ever, a good idea. The problems can be boiled down to five major points:

1) Naming conflicts between DroboApps: which python, for example, should be linked as /usr/bin/python if both the python2 and python3 DroboApps are installed?

2) Version incompatibilities: this one is usually more of a problem with libraries, but can (and does happen) with executables as well. If a new, incompatible version of some utility is published, how do other components that depend on that symlink will behave? The solution I have found so far is to create a separate DroboApp (e.g., python2 and python3) but that leads us back to the first point.

3) DroboApps are not a permanent part of the system. They can be very easily added and removed, which makes them very poor candidates for inclusion on the rootfs. To give you an example, I managed to almost lock myself out when I changed my shell to bash, forgot about it, and then inadvertently removed the bash DroboApp. Who knew that if the shell binary assigned to your account is missing it means locked account, eh? Hopefully, a simple firmware reset got me back on my feet. Which leads to point (4)...

4) Generally speaking, changes to the firmware are temporary at best. Every firmware upgrade wipes out any modifications to the rootfs, and there is no clear mechanism with which DroboApps can persistently make changes to the rootfs. In fact, making changes to rootfs is generally frowned upon, as explained in point (5).

5) All of the Drobo's storage areas that are not the home or share directories are under very strict control. I can tell you from personal experience that having a full rootfs, /var, or /tmp is *not* a fun situation to be in. Some of these will require a serial console (with all disassembling fun that comes with it) to recover from.

With all of that being said, the situation is much less grim that you'd expect. There are two ways that you can handle this in a somewhat predictable and persistent fashion.

The first way is to create a ~/bin directory and symlink all the binaries that you need there. Since user's home directories are on the diskpack, you can make as many changes as you want without any trouble. After that you just need to edit your .profile to include ~/bin on your PATH.

The second way is a bit more convoluted, but has the advantage of solving the problem for good. Edit your ~/.profile and include a little bit of scripting to add all the existing DroboApp bin to your PATH. Something like this:
Code:
for APP in /mnt/DroboFS/Shares/DroboApps/*; do
    if [ -d "$APP/bin" ]; then
        if [ "$PATH" == "${PATH/$APP/}" ]; then
            export PATH=$APP/bin:$PATH
        fi
    fi
done

This second way is what I do, and I had no trouble ever since I started using it.

Tap the full potential of the Drobo FS/5N with DroboApps.
To receive the latest updates about my DroboApps circle me on Google Plus.
Find all posts by this user
Quote this message in a reply
07-20-2016, 10:36 PM
Post: #3
RE: global symlinks to executables
Ricardo, thanks so much for taking the time to drop the knowledge. That is some supremely useful information!

Re: points 1,2 -- Yup, I anticipated that sort of thing.

Re: point 3 -- Agreed. (I've shot myself in the foot like that before, but not on a Drobo, so it was easier to recover!) Note that bash remains symlinked:

Code:
Drobo5N:~ $ ls -l /bin/bash
lrwxrwxrwx    1 root     root            43 May  3 19:05 /bin/bash -> /mnt/DroboFS/Shares/DroboApps/bash/bin/bash*

Re: point 4 -- Interesting. I was not aware that the rootfs got wiped during firmware upgrades, but it makes sense. However.... there are apps that make changes to the rootfs: bash and locales, for example. Does that mean that after each firmware update those apps are effectively broken? (They're an installed app, but their rootfs mods would be missing.)

Re: point 5 -- Well, "strict control" insofar as: 1) requires su privileges, 2) the rootfs gets wiped during firmware upgrade, and 3) the approval process for DroboApp presumably includes reviewing any changes to the rootfs preventing the release of a broken/dangerous app. Right?

Thanks for sharing your script to configure your PATH -- I was envisioning something like that. Only minor downside is that you can't control the order. (Not a big deal.)

Though the fact that this exercise is "left to the student" is somewhat disappointing/frustrating for newbies to the DroboApp platform -- I don't remember seeing it mentioned or hinted at. (Sorry if I missed it somewhere.)

Another related stumbling block: I'm trying to set up `man` to find the man pages for the installed DroboApps. (Setting MANPATH doesn't help.) I'm new to busybox, it's late, and I'm tired. Will poke around more tomorrow.
Find all posts by this user
Quote this message in a reply
07-21-2016, 09:48 AM
Post: #4
RE: global symlinks to executables
(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Re: point 4 -- Interesting. I was not aware that the rootfs got wiped during firmware upgrades, but it makes sense. However.... there are apps that make changes to the rootfs: bash and locales, for example. Does that mean that after each firmware update those apps are effectively broken? (They're an installed app, but their rootfs mods would be missing.)

If you look at the service.sh script on those apps you'll see that they effectively re-install themselves on each startup. It works, but it is not something that you'd rely on.

(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Re: point 5 -- Well, "strict control" insofar as: 1) requires su privileges, 2) the rootfs gets wiped during firmware upgrade, and 3) the approval process for DroboApp presumably includes reviewing any changes to the rootfs preventing the release of a broken/dangerous app. Right?

All of those and the fact that the system processes go to extreme lengths to make sure that none of those partitions runs out of space.

(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Thanks for sharing your script to configure your PATH -- I was envisioning something like that. Only minor downside is that you can't control the order. (Not a big deal.)

Well observed. That is why my approach is actually a hybrid of the two ways I presented. I have a ~/bin directory with symlinks to the versions that I want to ensure are found first, and I add ~/bin to PATH before the PATH enumeration script.

(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Though the fact that this exercise is "left to the student" is somewhat disappointing/frustrating for newbies to the DroboApp platform -- I don't remember seeing it mentioned or hinted at. (Sorry if I missed it somewhere.)

I can't really blame Drobo for that. The Drobo NAS were never meant to be full-fledged Linux boxes. I've noticed that Drobo is moving towards making the Linux side more user-friendly, but their primary goal is simplicity over power-users. I'm ok with that. When I want to experiment with some Linux app I use my Raspberry Pi. If the experiment pans out I move it to the Drobo.

(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Another related stumbling block: I'm trying to set up `man` to find the man pages for the installed DroboApps. (Setting MANPATH doesn't help.) I'm new to busybox, it's late, and I'm tired. Will poke around more tomorrow.

Although most DroboApps include manpages as a side-effect of the build process, I wouldn't bother too much. The path of least resistance is to google for the man page, which is what I do.

Tap the full potential of the Drobo FS/5N with DroboApps.
To receive the latest updates about my DroboApps circle me on Google Plus.
Find all posts by this user
Quote this message in a reply
07-27-2016, 11:04 AM
Post: #5
RE: global symlinks to executables
Sorry for the late reply. Life got in the way of my Drobo hacking.

(07-21-2016 09:48 AM)ricardo Wrote:  
(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Re: point 5 -- Well, "strict control" insofar as: 1) requires su privileges, 2) the rootfs gets wiped during firmware upgrade, and 3) the approval process for DroboApp presumably includes reviewing any changes to the rootfs preventing the release of a broken/dangerous app. Right?

All of those and the fact that the system processes go to extreme lengths to make sure that none of those partitions runs out of space.

Interesting. I'll have to poke around and see what "extreme lengths" you're referring to.

(07-21-2016 09:48 AM)ricardo Wrote:  
(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Though the fact that this exercise is "left to the student" is somewhat disappointing/frustrating for newbies to the DroboApp platform -- I don't remember seeing it mentioned or hinted at. (Sorry if I missed it somewhere.)

I can't really blame Drobo for that. The Drobo NAS were never meant to be full-fledged Linux boxes. I've noticed that Drobo is moving towards making the Linux side more user-friendly, but their primary goal is simplicity over power-users. I'm ok with that. When I want to experiment with some Linux app I use my Raspberry Pi. If the experiment pans out I move it to the Drobo.

Fair enough. One is bound to get cut on the bleeding edge.

(07-21-2016 09:48 AM)ricardo Wrote:  
(07-20-2016 10:36 PM)justin@worksperfectly.net Wrote:  Another related stumbling block: I'm trying to set up `man` to find the man pages for the installed DroboApps. (Setting MANPATH doesn't help.) I'm new to busybox, it's late, and I'm tired. Will poke around more tomorrow.

Although most DroboApps include manpages as a side-effect of the build process, I wouldn't bother too much. The path of least resistance is to google for the man page, which is what I do.

Yeah, Google FTW, I suppose.

That said, I do find it hard to keep straight which version/flavor of the tool I'm using on which system. A quick peek at the man usually jogs my memory as to the differences w/r/t versions or flavors (GNU vs BSD, etc.).

If/when I get and other related annoyances sorted out, I'll summarize it.

Thanks again.
Find all posts by this user
Quote this message in a reply
07-27-2016, 08:30 PM
Post: #6
RE: global symlinks to executables
So I just learned myself some BusyBox and now Ricardo's comments make more sense.

The man page thing is interesting. While the Drobo firmware has the man command included with the busybox binary, it doesn't seem to have the other commands necessary to display and format traditional man pages (nroff/troff). What a tease!

Indeed, if I set MANPATH to a known man dir, like so:

Code:
Drobo5N:/mnt/DroboFS/Shares/DroboApps/htop/man $ MANPATH=`pwd` man htop

man then executes and dumps me into a less session with "sh: nroff: not found". Undecided Case closed.

And I now see the rootfs only has 30mb, of which only 5mb is free.

Code:
Drobo5N:~ $ df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                   30.0M     24.8M      5.2M  83% /
/dev/root                30.0M     24.8M      5.2M  83% /
tmpfs                   433.6M     37.5M    396.1M   9% /tmp
/dev/shm                 64.0M     16.0K     64.0M   0% /dev/shm
/dev/mtdblock1            3.0M    592.0K      2.4M  19% /var
/dev/sda1                16.0T      3.2T     12.8T  20% /mnt/DroboFS

I yet didn't take a peek at what exactly the system is doing to keep free space available. (I assume something along the lines of aggressive logrotate configs.)
Find all posts by this user
Quote this message in a reply
07-28-2016, 11:33 AM
Post: #7
RE: global symlinks to executables
(07-27-2016 08:30 PM)justin@worksperfectly.net Wrote:  And I now see the rootfs only has 30mb, of which only 5mb is free.

Yep. From what I can tell rootfs and /var are on a flash memory chip on the board. There is a second copy of the rootfs there as well, but it is currently not used.

(07-27-2016 08:30 PM)justin@worksperfectly.net Wrote:  I yet didn't take a peek at what exactly the system is doing to keep free space available. (I assume something along the lines of aggressive logrotate configs.)

Bingo! They have very aggressive logrotate configs, and a background process that is constantly checking that log files are not running wild. Other things I noticed:

- All of the system processes' non-persistent data are stored either in the diskpack or in tmpfs. Check /tmp/DroboApps and /mnt/DroboFS/SystemLogs.

- All of the system processes take extreme care to clean up after themselves after a space-intensive operation like, for example, taking diags.

- Have a look at /etc/init.d/enable_var for another example of space cleanup tasks.

Tap the full potential of the Drobo FS/5N with DroboApps.
To receive the latest updates about my DroboApps circle me on Google Plus.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: