Troubles running Capstan

[aleks-f]

I'm following instructions on OS X Yosemite:

$ brew install qemu
$ curl https://raw.githubusercontent.com/cloudius-systems/capstan/master/scripts/download | bash
$ capstan run cloudius/osv

The documentation says that qemu is default but it looks for VBox nevertheless:

exec: "VBoxManage": executable file not found in $PATH

So, I force it to qemu:

$ capstan run -p qemu cloudius/osv

but it still complains:

No QEMU installation found. Use the CAPSTAN_QEMU_PATH environment variable to specify its path. 

OK, here:

$ export CAPSTAN_QEMU_PATH=/usr/local/bin

So, when I run it now , I get:

$ capstan run -p qemu cloudius/osv
Created instance: cloudius-osv

fork/exec /usr/local/bin: permission denied

Where am I going wrong? I own all the files/directories and all permissions on /usr/local/bin are 755.

EDIT: I tried installing from sources with same results:

$ /Users/alex/go/bin/capstan run cloudius/osv
Created instance: cloudius-osv

exec: "VBoxManage": executable file not found in $PATH

$ /Users/alex/go/bin/capstan run -p qemu cloudius/osv
Created instance: cloudius-osv

fork/exec /usr/local/bin: permission denied

EDIT2: Tried also Fedora and Ubuntu following instructions with no luck:

$ capstan run cloudius/osv
Created instance: cloudius-osv
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory

exit status 1

[nyh]

On Wed, Apr 29, 2015 at 6:21 AM, Aleksandar Fabijanic <
[email protected]> wrote:

No QEMU installation found. Use the CAPSTAN_QEMU_PATH environment variable
to specify its path.

OK, here:

$ export CAPSTAN_QEMU_PATH=/usr/local/bin

I think it expects this variable to contain the full pathname of the qemu
executable itself, not just the directory which contains it.

So, when I run it now , I get:

$ capstan run -p qemu cloudius/osv
Created instance: cloudius-osv

fork/exec /usr/local/bin: permission denied

As you can see, it tries to run $CAPSTAN_QEMU_PATH itself :-) So just set
it to the actual qemu executable's path.

Nadav Har'El
[email protected]

[dorlaor]

On Wed, Apr 29, 2015 at 6:21 AM, Aleksandar Fabijanic <
[email protected]> wrote:

I'm following instructions
https://github.com/cloudius-systems/capstan/blob/master/README.md on OS
X Yosemite:

$ brew install qemu
$ curl https://raw.githubusercontent.com/cloudius-systems/capstan/master/scripts/download | bash
$ capstan run cloudius/osv

The documentation says that qemu is default but it looks for VBox
nevertheless:

It's because on OSx we expect vbox only. Even if qemu will work it will be
dead slow since OSx doesn't support KVM. My recommendation is to try it w/
vbox.

exec: "VBoxManage": executable file not found in $PATH

So, I force it to qemu:

$ capstan run -p qemu cloudius/osv

but it still complains:

No QEMU installation found. Use the CAPSTAN_QEMU_PATH environment variable to specify its path.

OK, here:

$ export CAPSTAN_QEMU_PATH=/usr/local/bin

So, when I run it now , I get:

$ capstan run -p qemu cloudius/osv
Created instance: cloudius-osv

fork/exec /usr/local/bin: permission denied

Where am I going wrong? I own all the files/directories and all
permissions on /usr/local/bin are 755.

—
Reply to this email directly or view it on GitHub
#143.

[aleks-f]

Thanks, +1 for @nyh's answer, it works (duh for me).

@dorlaor I have installed vbox (using brew) but it does not seem to work:

$ capstan run cloudius/osv
Created instance: cloudius-osv

dial unix /Users/alex/.capstan/instances/vbox/cloudius-osv/cloudius-osv.sock: no such file or directory

[aleks-f]

OK, so I can run bare bone OSv on OSX (only using qemu) but when I try to run the capstan-example, here's what I get:

$ capstan run -p qemu
Created instance: capstan-example
OSv v0.20
eth0: 192.168.122.15
bad elf header. Powering off.

Note that I have compiled hello.so on Linux (and commented the make line in the Capstanfile on Mac):

$ file hello.so 
hello.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped

What am I doing wrong?

[nyh]

On Thu, Apr 30, 2015 at 2:20 AM, Aleksandar Fabijanic <
[email protected]> wrote:

OK, so I can run bare bone OSv on OSX (only using qemu) but when I try to
run the capstan-example, here's what I get:

$ capstan run -p qemu
Created instance: capstan-example
OSv v0.20
eth0: 192.168.122.15
bad elf header. Powering off.

Note that I have compiled hello.so on Linux (and commented the make line
in the Capstanfile on Mac):

$ file hello.so
hello.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped

The file type looks correct. Unfortunately, the "bad elf header" message
doesn't include the file's name (it should...) so I don't know if it really
tried to load this hello.so. My hunch is that it didn't, and instead tried
to run some non-object file (e.g., a directory...) - or, alternatively (?),
that Capstan copies some additional shared-libraries from your system to
the image, and those aren't Linux elf?

I'm really no Capstan expert (@penberg, comments?) - what is
"capstan-example" and does it really try to run this hello.so? Does it try
to include any other shared libraries in the image?

[dorlaor]

On Thu, Apr 30, 2015 at 10:42 AM, nyh [email protected] wrote:

On Thu, Apr 30, 2015 at 2:20 AM, Aleksandar Fabijanic <
[email protected]> wrote:

OK, so I can run bare bone OSv on OSX (only using qemu) but when I try to
run the capstan-example, here's what I get:

$ capstan run -p qemu
Created instance: capstan-example
OSv v0.20
eth0: 192.168.122.15
bad elf header. Powering off.

Note that I have compiled hello.so on Linux (and commented the make line
in the Capstanfile on Mac):

$ file hello.so
hello.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV),
dynamically linked, not stripped

The file type looks correct. Unfortunately, the "bad elf header" message
doesn't include the file's name (it should...) so I don't know if it really
tried to load this hello.so. My hunch is that it didn't, and instead tried
to run some non-object file (e.g., a directory...) - or, alternatively (?),
that Capstan copies some additional shared-libraries from your system to
the image, and those aren't Linux elf?

I'm really no Capstan expert (@penberg, comments?) - what is
"capstan-example" and does it really try to run this hello.so? Does it try
to include any other shared libraries in the image?

I'm no ELF expert but I can tell that Capstan image creation scenarios
were originally meant to be run on Linux only. The shared library is
supposed
to be created on Linux and executed within OSv. There is probably difference
on OSx. I bet it can be mitigated somehow. Worth to try the java example
of Capstan since it's suppose to be cross arch

—
Reply to this email directly or view it on GitHub
#143 (comment)
.

[penberg]

The "capstan-example" project is a C application. You cannot build in on OS X because its toolchain builds Mach-O object files, not ELF object files like Linux does.

[nyh]

On Wed, May 13, 2015 at 5:36 PM, Pekka Enberg [email protected]
wrote:

The "capstan-example" project is a C application. You cannot build in on
OS X because its toolchain builds Mach-O object files, not ELF object files
like Linux does.

I think he said he compiled the C file on Linux, and then copied over the
already compiled ".so" for Capstan to copy. But perhaps he missed
something, and Capstan compiled it again or something.

Nadav Har'El
[email protected]

[aleks-f]

Yes, as stated above, I have compiled Capstan on OSX but the hello.so is compiled on Linux and is ELF:

$ file hello.so 
hello.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped

But when I try to run:

$ capstan run -p qemu
Created instance: capstan-example
OSv v0.20
eth0: 192.168.122.15
bad elf header. Powering off.

Capstanfile:

base: cloudius/osv-base
cmdline: /tools/hello.so
#build: make
files:
  /tools/hello.so: hello.so

[gaffo]

@nyh what qemu binary are you pointing at? When installed with homebrew there is no actual qemu binary, there are:

>ls /usr/local/Cellar/qemu/2.4.0.1/bin/
qemu-img                 qemu-system-alpha        qemu-system-lm32         qemu-system-mips         qemu-system-moxie        qemu-system-ppcemb       qemu-system-sparc        qemu-system-x86_64
qemu-io                  qemu-system-arm          qemu-system-m68k         qemu-system-mips64       qemu-system-or32         qemu-system-s390x        qemu-system-sparc64      qemu-system-xtensa
qemu-nbd                 qemu-system-cris         qemu-system-microblaze   qemu-system-mips64el     qemu-system-ppc          qemu-system-sh4          qemu-system-tricore      qemu-system-xtensaeb
qemu-system-aarch64      qemu-system-i386         qemu-system-microblazeel qemu-system-mipsel       qemu-system-ppc64        qemu-system-sh4eb        qemu-system-unicore32

[nyh]

@gaffo, qemu-system-x86_64

But I don't understand what this issue is about...

[gaffo]

how am I supposed to build and run on Macos a c++ executable? My end goal is to get the capstan-examples c++ hello world running on my box. The errors I was getting were like above. Default on Macos seems to be qemu.

[gaffo]

So solution I found was following:

-rm -rf ~/.capstan
-delete all virts in virtualbox created by this madness
-git clone capstan-examples (if not done already)
-build the .so file with EITHER a virtual linux or cross compilation (trying this next maybe). I went with a ubuntu 14.4 server with a mapped directory under vagrant which landed me a hello.so file in my CWD
-capstan build -v -p vbox

Building ws-uk...
OSv v0.23
eth0: 192.168.122.15
Waiting for connection from host...

(process:22549): GLib-WARNING **: gmem.c:482: custom memory allocation vtable not supported
Uploading files...
hello.so  --> /tools/hello.so
Adding /tools/hello.so...

• capstan run -v -p vbox

➜  ws-uk  capstan run -v -p vbox
Created instance: ws-uk
OSv v0.23
eth0: 10.0.2.15
Hello, world!

If you build with the mac and no cross compile (default if you just run capstan examples, you'll have the wrong format for your .so file:

➜  ws-uk  capstan run -v -p vbox
Created instance: ws-uk
OSv v0.23
eth0: 10.0.2.15
bad elf header. Powering off.

Which means you need to either change make file to do a cross compile or go to a linux virt and make clean && make

Bad elf header basically means that it tried to run as a linux shared object (ELF) and instead got a mac so file which I don't know what that is.

[huntc]

It is unfortunate that the docs here are trailing behind somewhat - particularly given that this issue was reported back in April and it is now December. This doesn't provide me with a high level of confidence that the project maintainers understand many developers to be OS X based, and curious to see the JVM support happen.

After exporting the following:

export CAPSTAN_QEMU_PATH=/usr/local/bin/qemu-system-x86_64

...and then running:

capstan run target/capstan-example-java-1.0-SNAPSHOT-jar-with-dependencies.jar 
Building capstan-example-java-1.0-SNAPSHOT-jar-with-dependencies...
Uploading files...
1 / 1 [======================================================================================================================================================] 100.00 % Created instance: capstan-example-java-1.0-SNAPSHOT-jar-with-dependencies

VBoxManage: error: Cannot register the hard disk '/Users/huntc/.capstan/repository/capstan-example-java-1.0-SNAPSHOT-jar-with-dependencies/capstan-example-java-1.0-SNAPSHOT-jar-with-dependencies.vbox' {591053d7-030f-4afb-b3a2-c5143b8fda08} because a hard disk '/Users/huntc/.capstan/repository/capstan-example-java/capstan-example-java.vbox' with UUID {591053d7-030f-4afb-b3a2-c5143b8fda08} already exists
VBoxManage: error: Details: code NS_ERROR_INVALID_ARG (0x80070057), component VirtualBoxWrap, interface IVirtualBox, callee nsISupports
VBoxManage: error: Context: "OpenMedium(Bstr(pszFilenameOrUuid).raw(), enmDevType, enmAccessMode, fForceNewUuidOnOpen, pMedium.asOutParam())" at line 178 of file VBoxManageDisk.cpp

...not a great experience.

Why does it try to resolve anything re QEMU anyhow? I don't want QEMU - I want VirtualBox - which is what it is trying to do.

[nyh]

I don't know anything about Capstan & Vbox - @penberg might be able to help. But for what its worth, the error message doesn't seem to have anything to do with QEMU... It seems you created two images with identical UUIDs (unique identifiers) and Virtualbox didn't like it. Because as I said I never tried using Capstan with VirtualBox, I don't know why this happened, or how to work around it, but I'm quite sure it has nothing to do with QEMU.

[huntc]

Rewinding a bit, it asked for the QEMU env var before I provided it, despite targeting vbox. That at least doesn't seem right.

[mzagar]

@nyh I also encountered the

"Cannot register the hard disk '/path/to/osv/vbox/image/i/try/to/run' {UUID} because a hard disk '/path/to/another/osv/image/i/run/earlier' with UUID {UUID} already exists

There's a lot of info on the net how to work around this issue, I used this one (using mac) to change hard disk UUID of the image being reported in error above:
$ VBoxManage internalcommands sethduuid "/path/to/osv/vbox/image/i/try/to/run"

Not really sure what is causing this duplicate hard disk UUID or if it could somehow be prevented when capstan builds/runs virtualbox image.