Status of Queries not updating correctly after failure

[stvoutsin]

Description:

Status of a Query does not correctly get updated after it fails. This issue seems to be a one off, caused by an error in the communication between OGSA-DAI and the SQL database it is connecting to. The history of the query shows a "state = FAILED", but the status of the job remains as "RUNNING".
This has an effect of the Phase of the TAP service remaining as "Executing", so the client keeps querying Phase indefinitely (until it times out).

Example Query:
http://tap.roe.ac.uk/firethorn/blue/query/1775788

Query: SELECT TOP 1 * FROM TAP_SCHEMA.tables

OGSA-DAI Logs:

2022-02-15 13:15:36,167 DEBUG sql.SQLQueryActivity [pool-1-thread-152,cleanUp:517] Releasing Connection threw an Exception [[1644930936167:5085] uk.org.ogsadai.JDBC_CONNECTION_USE_ERROR : jdbc:postgresql://carolina:5432/postgres]
2022-02-15 13:15:36,168 DEBUG event.LoggingActivityListener [pool-1-thread-152,debug:84] uk.org.ogsadai.activity.sql.ActivitySQLUserException: The activity encountered a problem while interacting with a relational database.
	at uk.ac.roe.wfau.firethorn.ogsadai.activity.server.sql.SQLQueryActivity.processIteration(SQLQueryActivity.java:325)
	at uk.org.ogsadai.activity.MatchedIterativeActivity.process(MatchedIterativeActivity.java:90)
	at uk.org.ogsadai.activity.event.EventfulActivity.process(EventfulActivity.java:78)
	at uk.org.ogsadai.activity.concurrency.ActivityProcessingTask.call(ActivityProcessingTask.java:81)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:335)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
	at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
	at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
	at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
	at uk.org.ogsadai.activity.sql.CallableStatement.call(CallableStatement.java:66)
	at uk.ac.roe.wfau.firethorn.ogsadai.activity.server.sql.SQLQueryActivity$ChaoticCallableStatement.call(SQLQueryActivity.java:434)
	at uk.ac.roe.wfau.firethorn.ogsadai.activity.server.sql.SQLQueryActivity$ChaoticCallableStatement.call(SQLQueryActivity.java:409)
	... 4 more
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
	at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
	at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
	at org.postgresql.core.PGStream.receiveChar(PGStream.java:293)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1952)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
	... 15 more