singularity build has failed

[PabloMosUU]

I am trying to install LaMachine using Singularity on a Linux system, and I got this error:

FATAL: While pulling library image: error fetching image: request did not succeed: Bad image reference provided container ref is not valid: LaMachine (400 Bad Request)

The singularity build has failed unfortunately. You have several options:

• Retry the bootstrap, possibly tweaking configuration options
• File a bug report on https://github.com/proycon/LaMachine/issues/

This looks like an error with the image, and not in my computer. Can you confirm this, please?

[proycon]

I'll have to look deeper into this but Singularity support in LaMachine is very experimental (because nobody uses it yet afaik), so it's indeed quite possible that there is something wrong.

[proycon]

If docker or LXC is an option for you then I would recommend that for the time being.

[PabloMosUU]

Docker is not an option, unfortunately. What is LXC?

[proycon]

It's another container solution: https://linuxcontainers.org/#LXD .

But I'm willing to dive deeper into the Singularity problem too and make it work, you're the first user I think ;) But it'll be a bit delayed because of other priorities currently.

[PabloMosUU]

The situation is: I'm working on a cluster system where I am only allowed to install things via a small number of paths:

• Anaconda virtual environments
• Guix (similar to apt, but with a very limited list of available packages)
• Singularity
  I don't have root access in any case. I tried to install docker in a Guix profile, and there use the Docker version of LaMachine, but the bootstrap script did not find Docker. I can attempt to install it from source in a Guix profile, but neither Vagrant nor Virtualbox are available there, so I'd have to install those from source as well, and I can imagine that this will quickly escalate to something unmanageable. So I tried Singularity, since that seemed simplest, but it didn't work, as I mentioned.
  I will ask the system administrators if they have LXD available.

[proycon]

I see the issue yeah.

I tried to install docker in a Guix profile, and there use the Docker version of LaMachine, but the bootstrap script did not find Docker.

The script simply looks for a docker in your $PATH so I don't really know why that fails. But if it that would work, docker relies on an underlying daemon with elevated permissions so you probably can't run that from your Guix profile.

If the system administrators don't offer docker then they probably won't do LXD either.

Another option is to use LaMachine's local installation option, but that requires some global dependencies to be available, also depending on what parts of LaMachine you are interested in. What kind of Linux distribution and version is running on your cluster?

I'll also bump up the priority of getting singularity fixed now that we have an actual user for that.

[proycon]

Singularity should be able to grab images from a docker repository so I implement that as the default route for singularity now. If you select singularity and "pre-built" image, then things should hopefully work now. If you don't want to use the bootstrap, all this in essence does is:

singularity pull docker://proycon/lamachine

and then you run with singularity run -u lamachine_latest.sif

If you're planning on running the webserver inside LaMachine, to make use of the webservices or FLAT for instance, then we may have to do some extra work to get that to work with singularity.

[PabloMosUU]

Hi, first of all thanks for your assistance.
I attempted to use the "singularity pull" command, but that created a cache in my home directory, and it turns out we have a relatively low capacity in our home directories. I tried disabling the cache and/or moving it, following these instructions:
https://singularity.lbl.gov/build-environment
but it didn't work.

Since you asked, what I am trying to use is Frog:
http://languagemachines.github.io/frog/
I could try to build that from source, but the instructions look frankly quite formidable:
https://frognlp.readthedocs.io/en/latest/installation.html
However, if that is the only choice, I can give it a try

The administrator has confirmed we do not have access to LXD

[proycon]

You can probably just make a singularity dir on a disk where you are not limited by quotas and symlink it from ~/.singularity. Now that singularity works I guess it's your best option.

I could try to build that from source, but the instructions look frankly quite formidable:
https://frognlp.readthedocs.io/en/latest/installation.html
However, if that is the only choice, I can give it a try

There's a bunch of dependencies involved which may make it a bit more challenging yes, and those in turn rely on some global dependencies (like gcc, make, autoconf, libicu-dev etc)

[PabloMosUU]

Excellent idea. I'm trying it now. I got this error:

FATAL: While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: Error writing blob: write /scratch/2301226/bundle-temp-524237627/oci-put-blob013454405: no space left on device

I suppose I need to look into where cache goes. I thought it would go in that .singularity directory, but there might be something else. I need to inspect more

[proycon]

It seems your disk is full yeah, I can't help much with that. The sif file I got from the docker image was 2.7GB, but uncompressed it'll be more like 10GB.

[PabloMosUU]

Actually we have plenty of space in the disk. It must be the case that it is storing temporary files in another disk where there is only limited space, i.e., /scratch. I'll have to inspect that