calico-stefcal.py : load and apply only works on last tile's solutions

[IanHeywood]

Setting solution mode to 'load and apply' causes trouble. The example below shows the corrected data and the G amplitudes (RR). The pattern seems to repeat and doesn't match what is seen when I do a solve and save. Maybe it's loading the solutions for the first tile and applying them to all tiles, but there's also some weird behaviour with odd vs even numbered tiles...

[modhurita]

I can't comment on your particular issue with g solutions, but I have also been having problems with loading and applying previously calculated and saved ifr solutions, as I have pointed out in this thread.

[o-smirnov]

Yeah, at the moment it will only work if you set solution interval to the entire MS. As things stand, it only saves the solution for the last tile. Have you got enough RAM to process everything in one gulp?

Been meaning to fix that for ages, maybe the time has come...

[IanHeywood]

I don't think I have enough RAM on any of my local machines, although expanding that is on the cards. Jake / Elwood can surely process the whole MS but I can't use anything with a GUI when I log into them remotely, and I'm finding the browser very useful trying to figure all this out.

Only processing the solutions from the last tile makes sense looking at those plots. It's not an odd vs even tiles issue, rather the last chunk to be processed is a shorter time interval than the tile size so the preceding ones are only getting corrected for that fraction.

[o-smirnov]

I need a better scheme for saving/loading solutions anyway, the current 10 minute load time is unacceptable.

I also need three more Gijses to implement all the things that need implementing...

[IanHeywood]

As a hack for this I was thinking of using TaQL to loop over intervals of data equal to the tile size and make a gain_i.cp table for each calibration iteration. I can then load and apply the solutions in a similar fashion. I can't go down the re-compute route as I want to manipulate the derived solutions and then apply them.

Can you foresee any showstoppers before I try this?

Cheers.

[o-smirnov]

No, should work perfectly well...

On Tue, Jun 23, 2015 at 1:10 AM IanHeywood [email protected] wrote:

As a hack for this I was thinking of using TaQL to loop over intervals of
data equal to the tile size and make a gain_i.cp table for each calibration
iteration. I can then load and apply the solutions in a similar fashion. I
can't go down the re-compute route as I want to manipulate the derived
solutions and then apply them.

Can you foresee any showstoppers before I try this?

Cheers.

—
Reply to this email directly or view it on GitHub
#15 (comment)
.