Self-hosted runner (AMD mi250 CI caller) #408
Triggered via workflow run
December 14, 2023 13:31
amyeroberts
amyeroberts
completed
9e5c28c
Status
Failure
Total duration
15m 40s
Artifacts
–
self-push-amd-mi250-caller.yml
on: workflow_run
AMD mi250
/
Check Runner Status
9s
Matrix: AMD mi250 / Check Runners
Matrix: AMD mi250 / Setup
Matrix: AMD mi250 / Model tests
AMD mi250
/
Send results to webhook
27s
Annotations
2 errors and 4 warnings
|
AMD mi250 / Check Runners (multi-gpu)
System.IO.IOException: No space left on device : '/opt/github-runners/github-runner1/_diag/Worker_20231214-133217-utc.log'
at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
at System.IO.Strategies.BufferedFileStreamStrategy.FlushWrite()
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Diagnostics.TextWriterTraceListener.Flush()
at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
at GitHub.Runner.Common.HostTraceListener.TraceEvent(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, String message)
at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut)
at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
Unhandled exception. System.IO.IOException: No space left on device : '/opt/github-runners/github-runner1/_diag/Worker_20231214-133217-utc.log'
at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
at System.IO.Strategies.BufferedFileStreamStrategy.FlushWrite()
System.IO.IOException: No space left on device : '/opt/github-runners/github-runner1/_diag/Worker_20231214-133217-utc.log'
at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Diagnostics.TextWriterTraceListener.Flush()
at System.IO.Strategies.BufferedFileStreamStrategy.FlushWrite()
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Diagnostics.TextWriterTraceListener.Flush()
at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
at System.Diagnostics.TraceSource.Flush()
at GitHub.Runner.Common.HostTraceListener.TraceEvent(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, String message)
at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
at GitHub.Runner.Common.Tracing.Error(Exception exception)
at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
at GitHub.Runner.Common.TraceManager.Dispose(Boolean disposing)
at GitHub.Runner.Common.TraceManager.Dispose()
at GitHub.Runner.Common.HostContext.Dispose(Boolean disposing)
at GitHub.Runner.Common.HostContext.Dispose()
at GitHub.Runner.Worker.Program.Main(String[] args)
|
|
AMD mi250 / Check Runners (single-gpu)
The job was canceled because "multi-gpu" failed.
|
|
AMD mi250 / Check Runners (multi-gpu)
You are running out of disk space. The runner will stop working when the machine runs out of disk space. Free space left: 45 MB
|
|
AMD mi250 / Check Runners (multi-gpu)
Delete stale container networks failed, docker network prune fail with exit code 1
|
|
AMD mi250 / Check Runners (single-gpu)
Runner hf-amd-mi250-ci-1gpu-2 did not respond to a cancelation request with 00:05:00.
|
|
AMD mi250 / Check Runners (single-gpu)
You are running out of disk space. The runner will stop working when the machine runs out of disk space. Free space left: 68 MB
|