Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
swarp breaks when coadding many overlapping files
01-24-2016, 17:11
Post: #1
swarp breaks when coadding many overlapping files
Hi again,

I just got another bug report from Julian Taylor, which I will just cut&paste here:

Quote:when coadding more than the openfile limit (normally 1024) overlapping files swarp aborts due to too many open files.
To reproduce create some files with overlapping wcs coordinates, set ulimit -n to a low enough number and just run SWarp <files>
This was fixed a while ago but apparently in 2.38 the original patch has
been mangled into not working anymore.

1. a in my opinion not useful NOPENFILES_MAX variable was added, and when you go over the limit it just warns instead of capping the number of files swarp uses (src/prefs.c:536). The original patch capped the usage and did not require user configuration or having the process abort after hours of resampling because the warning was not read.
The fix is simple, set the cap in src/prefs.c, as the actual meat of the original patch is still in place.

2. the cfitsio based opening leaks open file descriptors somewhere. The code is kind of convoluted so I could not figure out a good fix. This half backed attempt works for the visir pipeline but is likely not appropriate for upstream:
Code:
--- a/swarp/src/fits/fitscat.c
+++ b/swarp/src/fits/fitscat.c
@@ -161,6 +161,7 @@ int close_cat(catstruct *cat)
     }

   cat->file = NULL;
+  close_cfitsio(cat);

   return RETURN_OK;
   }
diff --git a/swarp/src/fits/fitstab.c b/swarp/src/fits/fitstab.c
index 5919b2a..95bb2c4 100644
--- a/swarp/src/fits/fitstab.c
+++ b/swarp/src/fits/fitstab.c
@@ -285,6 +285,9 @@ void        copy_tab_fromptr(tabstruct *tabin,
catstruct *catout, int pos)
/* First, allocate memory and copy data */
    QCALLOC(tabout, tabstruct, 1);
    *tabout = *tabin;
+   /* set fits handle to NULL to avoid double close and hope its not needed
+    * anymore */
+   tabout->infptr = NULL;
    if (tabin->naxis)
      QMEMCPY(tabin->naxisn, tabout->naxisn, int, tabin->naxis);
    if (tabin->headbuf)
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)