Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to run locally. #5

Open
shenwei356 opened this issue Mar 10, 2024 · 5 comments
Open

Failed to run locally. #5

shenwei356 opened this issue Mar 10, 2024 · 5 comments

Comments

@shenwei356
Copy link

Hi, I'm trying to install and run BigSeqKit locally.

Here are the commands. There's an error of Permission denied, but it remains after changing ownership of the ignis directory.

# no error
sudo docker pull ignishpc/full

cd ~/Downloads
# no error
docker run --rm -v $(pwd):/target ignishpc/submitter ignis-export /target

# set current directory as job directory
export IGNIS_DFS_ID=$(pwd)
# set docker as scheduler
export IGNIS_SCHEDULER_TYPE=docker
# set where docker is available
export IGNIS_SCHEDULER_URL=/var/run/docker.sock

./ignis/bin/ignis-submit ignishpc/full bigseqkit
Mar 10, 2024 09:03:37 <ERROR> org.ignis.submitter.Submit.call(Submit.java:271): java.io.IOException: com.sun.jna.LastErrorException: [13] Permission denied
java.lang.RuntimeException: java.io.IOException: com.sun.jna.LastErrorException: [13] Permission denied
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:179) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:9) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.DefaultInvocationBuilder.post(DefaultInvocationBuilder.java:124) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.exec.CreateContainerCmdExec.execute(CreateContainerCmdExec.java:37) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.exec.CreateContainerCmdExec.execute(CreateContainerCmdExec.java:13) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35) ~[docker-java-core-3.2.11.jar:?]
        at com.github.dockerjava.core.command.CreateContainerCmdImpl.exec(CreateContainerCmdImpl.java:609) ~[docker-java-core-3.2.11.jar:?]
        at org.ignis.scheduler.Docker.createDriverContainer(Docker.java:318) ~[ignis-scheduler-docker-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:186) [ignis-submitter-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:46) [ignis-submitter-1.0.jar:?]
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.access$1300(CommandLine.java:145) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.execute(CommandLine.java:2078) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at org.ignis.submitter.Submit.main(Submit.java:284) [ignis-submitter-1.0.jar:?]
Caused by: java.io.IOException: com.sun.jna.LastErrorException: [13] Permission denied
        at com.github.dockerjava.transport.DomainSocket.<init>(DomainSocket.java:63) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.LinuxDomainSocket.<init>(LinuxDomainSocket.java:41) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.get(DomainSocket.java:138) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl$2.createSocket(ApacheDockerHttpClientImpl.java:137) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:125) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:409) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:164) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:174) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:135) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:172) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:93) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:128) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:178) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:67) ~[httpclient5-5.0.3.jar:5.0.3]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:175) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        ... 19 more
Caused by: com.sun.jna.LastErrorException: [13] Permission denied
        at com.github.dockerjava.transport.LinuxDomainSocket.connect(Native Method) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.LinuxDomainSocket.connect(LinuxDomainSocket.java:49) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.open(DomainSocket.java:69) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.<init>(DomainSocket.java:59) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.LinuxDomainSocket.<init>(LinuxDomainSocket.java:41) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.transport.DomainSocket.get(DomainSocket.java:138) ~[docker-java-transport-3.2.11.jar:?]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl$2.createSocket(ApacheDockerHttpClientImpl.java:137) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:125) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:409) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:164) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:174) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:135) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:172) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:93) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:128) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:178) ~[httpclient5-5.0.3.jar:5.0.3]
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:67) ~[httpclient5-5.0.3.jar:5.0.3]
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:175) ~[docker-java-transport-httpclient5-3.2.11.jar:?]
        ... 19 more
@cesarpomar
Copy link
Member

The Permission denied error is because your user doesn't have permission to access /var/run/docker.sock. Please check if you can run docker ps.

Another option is to run ignis-submit command with sudo.

Please let me know if you find any other errors

@shenwei356
Copy link
Author 

$ sudo docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

$ sudo ./ignis/bin/ignis-submit ignishpc/full bigseqkit
Mar 12, 2024 09:59:28 <ERROR> org.ignis.submitter.Submit.call(Submit.java:271): Key 'ignis.scheduler.type' value error,  not found
org.ignis.properties.IPropertyException: Key 'ignis.scheduler.type' value error,  not found
        at org.ignis.properties.IProperties.getProperty(IProperties.java:69) ~[ignis-properties-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:113) [ignis-submitter-1.0.jar:?]
        at org.ignis.submitter.Submit.call(Submit.java:46) [ignis-submitter-1.0.jar:?]
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.access$1300(CommandLine.java:145) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at picocli.CommandLine.execute(CommandLine.java:2078) [picocli-4.6.1.jar:task ':jar' property 'archiveVersion']
        at org.ignis.submitter.Submit.main(Submit.java:284) [ignis-submitter-1.0.jar:?]

@cesarpomar
Copy link
Member

You should use sudo -E to preserve the environment variables defined with the export command. The error indicates that ignis.scheduler.type created with export IGNIS_SCHEDULER_TYPE=docker does not exist.

@shenwei356
Copy link
Author

Oh thanks, it finally worked. However a new error comes out. It reported a file not exist error. But the input file does exist.

$ ll /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa
-rw-r--r-- 1 shenwei shenwei 4.6M  9月25日 08:16 /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa

$ sudo -E sh ./ignis/bin/ignis-submit ignishpc/full 'bigseqkit seq /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa | bigseqkit stats'

...
panic: stat /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa: no such file or directory
...
details. 
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:46): Backend started
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:49): Loading environment variables
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:64): Loading configuration file
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:80): Loading scheduler
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:92): Checking scheduler docker
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:95): Scheduler docker /var/run/docker.sock ... OK
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:103): Getting Driver container info from scheduler
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:105): Driver container info found
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.Main.main(Main.java:125): Setting dynamic properties
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.services.IBackendServiceImpl.start(IBackendServiceImpl.java:48): Backend server started on port 4000
[INFO]  2024/03/12 14:14:35 ilog.go:37: ServerModule: go executor started
Mar 12, 2024 02:14:35 <INFO> org.ignis.backend.services.IBackendServiceImpl.startHealthServer(IBackendServiceImpl.java:117): Backend health server started
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterCreateHelper.create(IClusterCreateHelper.java:53): Cluster(0): Registering cluster with 1 containers
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.worker.IWorkerCreateHelper.create(IWorkerCreateHelper.java:57): Cluster(0), Worker(0): Registering worker with 1 executors with 1 cores
panic: stat /home/shenwei/go/src/github.com/shenwei356/seqkit/tests/hairpin.fa: no such file or directory

goroutine 1 [running]:
main.checkError(...)
    /tmp/src/bigseqkit-cli/helper.go:34
main.getFileList({0xc000344720, 0x3, 0x3}, 0x1)
    /tmp/src/bigseqkit-cli/helper.go:193 +0x105
main.getFileListFromArgsAndFile(0xc000722390?, {0xc000344720, 0x3, 0x3}, 0x0?, {0x12b5059?, 0xd7e260?}, 0x40?)
    /tmp/src/bigseqkit-cli/helper.go:231 +0x79
main.readSeqs(0xc0007280e8?, {0xc000344720?, 0x2?, 0x0?}, 0x4?)
    /tmp/src/bigseqkit-cli/helper.go:56 +0x9a
main.ignisDriver(0x108f840?, {0xc000344720, 0x3, 0x3}, 0x1305858)
    /tmp/src/bigseqkit-cli/helper.go:104 +0x315
main.init.17.func1.1(0xc000348000?, {0xc000344720?, 0x3?, 0x3?})
    /tmp/src/bigseqkit-cli/seq.go:48 +0x25
github.com/spf13/cobra.(*Command).execute(0xc000348000, {0xc0003446f0, 0x3, 0x3})
    /root/go/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x663
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000d4c80)
    /root/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
    /root/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
main.main()
    /tmp/src/bigseqkit-cli/main.go:4 +0x1e
Mar 12, 2024 02:14:36 <WARN> org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:159): Error closing output stream.
java.net.SocketException: Socket closed
    at sun.nio.ch.NioSocketImpl.ensureOpenAndConnected(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl.beginWrite(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl.implWrite(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl.write(Unknown Source) ~[?:?]
    at sun.nio.ch.NioSocketImpl$2.write(Unknown Source) ~[?:?]
    at java.net.Socket$SocketOutputStream.write(Unknown Source) ~[?:?]
    at java.io.BufferedOutputStream.flushBuffer(Unknown Source) ~[?:?]
    at java.io.BufferedOutputStream.flush(Unknown Source) ~[?:?]
    at java.io.FilterOutputStream.close(Unknown Source) ~[?:?]
    at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:157) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TSocket.close(TSocket.java:252) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TZlibTransport.close(TZlibTransport.java:83) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:267) [libthrift-0.15.0.jar:0.15.0]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
    at java.lang.Thread.run(Unknown Source) [?:?]
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.services.IBackendServiceImpl.stopAll(IBackendServiceImpl.java:86): Stopping Backend server
Mar 12, 2024 02:14:36 <WARN> org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:159): Error closing output stream.
java.io.IOException: org.apache.thrift.transport.TTransportException: Cannot write to null outputStream
    at org.apache.thrift.transport.TTransportOutputStream.write(TZlibTransport.java:139) ~[libthrift-0.15.0.jar:0.15.0]
    at java.util.zip.DeflaterOutputStream.deflate(Unknown Source) ~[?:?]
    at java.util.zip.DeflaterOutputStream.finish(Unknown Source) ~[?:?]
    at java.util.zip.DeflaterOutputStream.close(Unknown Source) ~[?:?]
    at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:157) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TZlibTransport.close(TZlibTransport.java:81) [libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:270) [libthrift-0.15.0.jar:0.15.0]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
    at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: org.apache.thrift.transport.TTransportException: Cannot write to null outputStream
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:194) ~[libthrift-0.15.0.jar:0.15.0]
    at org.apache.thrift.transport.TTransportOutputStream.write(TZlibTransport.java:137) ~[libthrift-0.15.0.jar:0.15.0]
    ... 9 more
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.services.IBackendServiceImpl.stopHealthServer(IBackendServiceImpl.java:127): Backend health server stopped
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.services.IBackendServiceImpl.start(IBackendServiceImpl.java:71): Backend server stopped
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterDestroyHelper.destroy(IClusterDestroyHelper.java:41): Cluster(0): Preparing cluster to destroy
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.worker.IWorkerDestroyHelper.destroy(IWorkerDestroyHelper.java:38): Cluster(0), Worker(0): Preparing worker to destroy
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterDestroyHelper.lambda$destroy$0(IClusterDestroyHelper.java:57): Cluster(0): Destroying cluster with 1 containers
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.cluster.helpers.cluster.IClusterDestroyHelper.lambda$destroy$0(IClusterDestroyHelper.java:59): Cluster(0): Cluster destroyed
Mar 12, 2024 02:14:36 <INFO> org.ignis.backend.Main.main(Main.java:176): Backend stopped

@cesarpomar
Copy link
Member

Using Docker containers, only the IGNIS_DFS_ID is mounted and set as the working directory. Therefore, you can reference files within that directory using relative paths.

You can use IgnisHPC properties to manually mount other directories using -p just after the script.

...ignis-submit -p ignis.driver.bind.{containerpath}={hostpath} -p ignis.executor.bind.{containerpath}={hostpath}...

We're working on a simpler implementation, but it's not ready yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@shenwei356 @cesarpomar