source: trunk/src/astrometry/README @ 20087

Revision 20087, 31.1 KB checked in by dstn, 5 months ago (diff)
Line 
1Astrometry.net code release
2===========================
3
4Copyright 2006-2010 Michael Blanton, David W. Hogg, Dustin Lang, Keir
5Mierle and Sam Roweis.
6Copyright 2011 Dustin Lang and David W. Hogg.
7
8This code is accompanied by the paper:
9
10Lang, D., Hogg, D. W.; Mierle, K., Blanton, M., & Roweis, S., 2010,
11Astrometry.net: Blind astrometric calibration of arbitrary
12astronomical images, Astronomical Journal 137, 1782–1800.
13http://arxiv.org/abs/0910.2233
14
15The original purpose of this code release was to back up the claims in
16the paper in the interest of scientific repeatability.  Over the
17years, it has become more robust and usable for a wider audience, but
18it's still neither totally easy nor bug-free.
19
20This release includes a snapshot of all of the components of our
21current research code, including routines to:
22  * Convert raw USNO-B and Tycho2 into FITS format for easier use
23  * Uniformize, deduplicate, and cut the FITSified catalogs
24  * Build index files from these cuts
25  * Solve the astrometry of images using these index files
26
27The code includes:
28  * A simple but powerful HEALPIX implementation
29  * The QFITS library with several modifications
30  * libkd, a compact and high-performance kdtree library
31
32In principle, the code in the tarball is sufficient to reproduce our
33entire astrometry system and replicate our blind astrometry results
34(e.g. on the SDSS fields).  However, this requires the entire USNO-B
351.0 catalogue which is about 80GB in size.  For ease of use, we are
36distributing our pre-cooked index files (about 25 GB total), which are
37the same files we're using for our web service.
38
39INSTALLING
40----------
41
42The short version:
43
44   make
45   make extra
46   make install  # to put it in /usr/local/astrometry
47   # or:
48   make install INSTALL_DIR=/some/other/place
49
50
51The long version:
52
53For full functionality, you will need:
54  * GNU build tools (gcc, make, etc.)
55  * cairo
56  * netpbm
57  * libpng
58  * libjpeg
59  * libz
60  * python (probably >= 2.4)
61  * numpy
62  * pyfits
63      http://www.stsci.edu/resources/software_hardware/pyfits
64
65-> On Ubuntu or Debian-like systems, the following should suffice:
66
67    $ sudo apt-get install libcairo2-dev libnetpbm10-dev netpbm \
68                           libpng12-dev libjpeg-dev python-numpy \
69                                                   zlib1g-dev python-pyfits
70
71-> On RHEL, you'll need something like:
72    $ sudo yum install cairo.x86_64 cairo-devel.x86_64 netpbm.x86_64 \
73                       netpbm-devel.x86_64 fontconfig-devel.x86_64 \
74                       libXrender-devel.x86_64 xorg-x11-proto-devel.x86_64 \
75                                           zlib-devel
76   On Fedora, you may also need:
77    $ sudo yum install netpbm-progs
78
79-> On Mac OS X, possibly the easiest way is to install Fink
80   (www.finkproject.org), and then use apt-get install as per the
81   Debian instructions above (leaving out "zlib1g-dev" because it's
82   already included with OSX).  Note that to use Fink you will need to
83   add something like this in your ~/.profile or ~/.bashrc file:
84
85. /sw/bin/init.sh
86export CFLAGS="-I/usr/local/include -I/sw/include"
87export LDFLAGS="-L/usr/local/lib -L/sw/lib"
88
89
90If you don't have and can't get these libraries, you should still be
91able to compile and use the core parts of the solver, but you will
92miss out on some eye-candy.
93
94Grab the code:
95  $ wget http://astrometry.net/downloads/astrometry.net-0.38.tar.bz2
96  $ tar xjf astrometry.net-0.38.tar.bz2
97  $ cd astrometry.net-0.38
98
99Build the solving system:
100  $ make
101
102If you installed the libraries listed above, build the plotting code:
103  $ make extra
104
105Install it:
106  $ make install
107
108You might see some error message during compilation; see the section
109ERROR MESSAGES below for fixes to common problems.
110
111By default it will be installed in  /usr/local/astrometry .
112You can override this by either:
113     * editing the top-level Makefile (look for INSTALL_DIR)
114  or * defining INSTALL_DIR on the command-line:
115        For bash shell:
116          $ export INSTALL_DIR=/path/to/astrometry
117          $ make install
118        or
119          $ INSTALL_DIR=/path/to/astrometry make install
120        For tcsh shell:
121          $ setenv INSTALL_DIR /path/to/astrometry
122          $ make install
123
124The astrometry solver is composed of several executables.  You may
125want to add the INSTALL_DIR/bin directory to your path:
126   For bash shell:
127     $ export PATH="$PATH:/usr/local/astrometry/bin"
128   For tcsh shell:
129     $ setenv PATH "$PATH:/usr/local/astrometry/bin"
130
131
132GETTING INDEX FILES
133-------------------
134
135Next, grab some pre-cooked index files.  See the file GETTING-INDEXES
136for details about how to do this.
137
138(If you have index files from version 0.1 of the Astrometry.net
139software, see the "Upgrading" section below.)
140
141Each index file is designed to solve images within a narrow range of
142scales.  The index files designed to solve small (angular size) images
143are rather large in file size, so you probably only want to grab the
144index files required for the range of image scales you wish to solve.
145If you grab extra index files, the solver will run more slowly, but
146the results should be the same.
147
148Each index contains a large number of "skymarks" (landmarks for the
149sky) that allow our solver to identify your images.  The skymarks
150contained in each index have sizes (diameters) within a range of
151scales (a factor of sqrt(2)).  You probably want to download indexes
152whose quads are, say, 10% to 100% of the sizes of the images you want
153to solve.
154
155For example, let's say you have some 1-degree square images.  You
156should grab index files that contain skymarks of size 0.1 to 1 degree,
157or 6 to 60 arcminutes.  Referring to the table below, you should grab
158index files 203 through 209.  You might find that the same number of
159fields solve, and faster, using just one or two of the index files in
160the middle of that range - in our example you might try 205, 206 and
161207.
162
163For reference, we used index 202 alone for our SDSS tests (13x9 arcmin
164fields).
165
166---------------------------------------------------------------
167Index Filename          Range of skymark diameters    File Size
168                               (arcminutes)              (MB)
169---------------------------------------------------------------
170index-210+.tar.bz2           60  - 2000                    29
171index-209.tar.bz2            42  -   60                    30
172index-208.tar.bz2            30  -   42                    61
173index-207.tar.bz2            22  -   30                   125
174index-206.tar.bz2            16  -   22                   255
175index-205.tar.bz2            11  -   16                   526
176index-204.tar.bz2             8  -   11                 1,023
177index-203.tar.bz2             5.6  -  8.0               2,089
178index-202.tar.bz2             4.0  -  5.6               4,260
179index-201.tar.bz2             2.8  -  4.0               5,878
180index-200.tar.bz2             2.0  -  2.8              10,058
181--------------------------------------------------------------
182
183Download the index files you need and put them in the top-level
184(astrometry-0.38) source directory.
185
186Install them:
187   $ make install-indexes
188
189(Or you can extract them into the INSTALL_DIR/data directory by hand.)
190
191
192Next, you can (optionally) configure the solver by editing the file
193   INSTALL_DIR/etc/backend.cfg
194
195
196AUTO-CONFIGURATION
197------------------
198
199We use a do-it-yourself auto-config system that tries to detect what
200is available on your machine.  It is called "os-features", and it
201works by trying to compile, link, and run a number of executables to
202detect:
203
204-whether the "netpbm" library is available
205-whether certain GNU-specific function calls exist
206
207You can change the flags used to compile and link "netpbm" by either:
208
209-editing util/makefile.netpbm
210-setting NETPBM_INC or NETPBM_LIB, like this:
211  $ make NETPBM_INC="-I/tmp" NETPBM_LIB="-L/tmp -lnetpbm"
212
213You can see whether netpbm was successfully detected by:
214
215  $ cat util/makefile.os-features
216  # This file is generated by util/Makefile.
217  HAVE_NETPBM := yes
218
219You can force a re-detection either by deleting util/makefile.os-features
220and util/os-features-config.h, or running:
221  $ make reconfig
222(which just deletes those files)
223
224
225UPGRADING INDEX FILES (only necessary if you have old old index files!)
226---------------------
227
228The format of the index files changed after the 0.11 release.  There
229is a program for updating them, and there is also a Makefile target.
230
231  $ make upgrade-indexes
232
233If you want to do it by hand, use the program "fix-bb".  Run it on
234each of the ckdt.fits and skdt.fits files in your INSTALL_DIR/data
235directory.
236
237
238BIG-ENDIAN MACHINES
239-------------------
240
241Most CPUs these days are little-endian.  If you have an Intel or AMD
242chip, you can skip this section.  The most common big-endian CPU in
243recent times is the PowerPC used in Macs.  If you have one of these,
244read on.
245
246The index files we are distributing are for little-endian machines.
247For big-endian machines, you must do the following:
248
249cd /usr/local/astrometry/data
250for f in index-*.fits; do
251  fits-flip-endian -i $f -o flip-$f -e 1 -s 4 -e 3 -s 4 -e 4 -s 2 -e 5 -s 8 -e 6 -s 2 -e 8 -s 4 -e 9 -s 4 -e 10 -s 8 -e 11 -s 4
252  for e in 0 2 7; do
253    modhead flip-$f"[$e]" ENDIAN 01:02:03:04
254  done
255done
256
257assuming "fits-flip-endian" and "modhead" are in your path.  The files
258"flip-index-*.fits" will contain the flipped index files.
259
260If that worked, you can swap the flipped ones into place (while saving
261the originals) with:
262
263cd /usr/local/astrometry/data
264mkdir -p orig
265for f in index-*.fits; do
266  echo "backing up $f"
267  mv -n $f orig/$f
268  echo "moving $f into place"
269  mv -n flip-$f $f
270done
271
272SOLVING
273-------
274
275Finally, solve some fields.
276
277(If you didn't build the plotting commands, add "--no-plots" to the
278command lines below.)
279
280If you have any of index files 213 to 218:
281   $ solve-field --scale-low 10 demo/apod4.jpg
282
283If you have index 219:
284   $ solve-field --scale-low 30 demo/apod5.jpg
285
286If you have any of index files 210 to 214:
287   $ solve-field --scale-low 1 demo/apod3.jpg
288
289If you have any of index files 206 to 211:
290   $ solve-field --scale-low 1 demo/apod2.jpg
291
292If you have any of index files 203 to 205:
293   $ solve-field apod1.jpg
294
295If you have any of index files 200 to 203:
296   $ solve-field demo/sdss.jpg
297
298
299Copyrights and credits for the demo images are listed in the file
300demo/CREDITS .
301
302Note that you can also give solve-field a URL rather than a file as input:
303   $ solve-field --out apod1b http://antwrp.gsfc.nasa.gov/apod/image/0302/ngc2264_croman_c3.jpg
304
305
306If you don't have the netpbm tools (eg jpegtopnm), do this instead:
307
308If you have any of index files 213 to 218:
309   $ solve-field --scale-low 10 demo/apod4.xyls
310
311If you have index 219:
312   $ solve-field --scale-low 30 demo/apod5.xyls
313
314If you have any of index files 210 to 214:
315   $ solve-field --scale-low 1 demo/apod3.xyls
316
317If you have any of index files 206 to 211:
318   $ solve-field --scale-low 1 demo/apod2.xyls
319
320If you have any of index files 203 to 205:
321   $ solve-field demo/apod1.xyls
322
323If you have any of index files 200 to 203:
324   $ solve-field demo/sdss.xyls
325
326
327OUTPUT FILES
328------------
329
330   <base>-ngc.png   : an annotation of the image.
331   <base>.wcs       : a FITS WCS header for the solution.
332   <base>.new       : a new FITS file containing the WCS header.
333   <base>-objs.png  : a plot of the sources (stars) we extracted from
334                      the image.
335   <base>-indx.png  : sources (red), plus stars from the index (green),
336                      plus the skymark ("quad") used to solve the
337                      image.
338   <base>-indx.xyls : a FITS BINTABLE with the pixel locations of
339                      stars from the index.
340   <base>.rdls      : a FITS BINTABLE with the RA,Dec of sources we
341                      extracted from the image.
342   <base>.axy       : a FITS BINTABLE of the sources we extracted, plus
343                      headers that describe the job (how the image is
344                      going to be solved).
345   <base>.solved    : exists and contains (binary) 1 if the field solved.
346   <base>.match     : a FITS BINTABLE describing the quad match that
347                      solved the image.
348   <base>.kmz       : (optional) KMZ file for Google Sky-in-Earth.  You need
349                      to have "wcs2kml" in your PATH.  See
350                       http://code.google.com/p/wcs2kml/downloads/list
351                       http://code.google.com/p/google-gflags/downloads/list
352
353
354COMMON ERRORS AND FIXES/WORKAROUNDS
355-----------------------------------
356
357Error 1. solve-field fails with:
358
359Traceback (most recent call last):
360File "/usr/local/astrometry/bin/removelines.py", line 12, in
361import pyfits
362ImportError: No module named pyfits
363augment-xylist.c:586:backtick Failed to run command: /usr/local/astrometry/bin/removelines.py /tmp/tmp.xyls.UjOdFo /tmp/tmp.removelines.8LHW36
364ioutils.c:669:run_command_get_outputs Command failed: return value 1
365
366Solution A: get the "pyfits" module and install it;
367  http://www.stsci.edu/resources/software_hardware/pyfits  .
368
369It must be installed so that this command runs without error:
370  python -c "import pyfits"
371
372Solution B: turn off "features" that require pyfits:
373
374  solve-field --uniformize 0 --no-remove-lines --no-fits2fits  [...]
375
376
377
378TRICKS AND TIPS
379---------------
380
381*** To lower the CPU time limit before giving up:
382
383    $  solve-field --cpulimit 30 ...
384
385will make it give up after 30 seconds.
386
387(Note, however, that the "backend" configuration file (backend.cfg)
388puts a limit on the CPU time that is spent on an image; solve-field
389can reduce this but not increase it.)
390
391
392*** "I know where my image is to within 1 arcminute, how can I tell
393solve-field to only look there?"
394
395   $ solve-field --ra, --dec, --radius
396
397Tells it to look within "radius" degrees of the given RA,Dec position.
398
399*** Scale of the image: if you provide bounds (lower and upper limits)
400on the size of the image you are trying to solve, solving can be much
401faster.  In the last examples above, for example, we specified that
402the field is at least 30 degrees wide: this means that we don't need
403to search for matches in the indexes that contain only tiny skymarks.
404
405Eg, to specify that the image is between 1 and 2 degrees wide:
406
407   $ solve-field --scale-units degwidth --scale-low 1 --scale-high 2 ...
408
409If you know the pixel scale instead:
410
411   $ solve-field --scale-units arcsecperpix \
412       --scale-low 0.386 --scale-high 0.406 ...
413
414When you tell solve-field the scale of your image, it uses this to
415decide which index files to try to use to solve your image; each index
416contains quads whose scale is within a certain range, so if these
417quads are too big or too small to be in your image, there is no need
418to look in that index.  It is also used while matching quads: a small
419quad in your image is not allowed to match a large quad in the index
420if such a match would cause the image scale to be outside the bounds
421you specified.  However, all these checks are done before computing a
422best-fit WCS solution and polynomial distortion terms, so it is
423possible (though rare) for the final solution to fall outside the
424limits you specified.  This should only happen when the solution is
425correct, but you gave incorrect inputs, so you shouldn't be
426complaining! :)
427
428
429*** Guess the scale: solve-field can try to guess your image's scale
430from a number of different FITS header values.  When it's right, this
431often speeds up solving a lot, and when it's wrong it doesn't cost
432much.  Enable this with:
433
434   $ solve-field --guess-scale ...
435
436
437*** Depth.  The solver works by looking at sources in your image,
438starting with the brightest.  It searches for all "skymarks" that can
439be built from the N brightest stars before considering star N+1.  When
440using several indexes, it can be much faster to search for many
441skymarks in one index before switching to the next one.  This flag
442lets you control when the solver switches between indexes.  It also
443lets you control how much effort the solver puts in before giving up -
444by default it looks at all the sources in your image, and usually
445times out before this finishes.
446
447Eg, to first look at sources 1-20 in all indexes, then sources 21-30
448in all indexes, then 31-40:
449
450    $ solve-field --depth 20,30,40 ...
451 or $ solve-field --depth 1-20 --depth 21-30 --depth 31-40 ...
452
453Sources are numbered starting at one, and ranges are inclusive.  If
454you don't give a lower limit, it will take 1 + the previous upper
455limit.  To look at a single source, do
456
457    $ solve-field --depth 42-42 ...
458
459
460*** Our source extractor sometimes estimates the background badly, so
461by default we sort the stars by brightness using a compromise between
462the raw and background-subtracted flux estimates.  For images without
463much nebulosity, you might find that using the background-subtracted
464fluxes yields faster results.  Enable this by:
465
466    $ solve-field --resort ...
467
468
469*** If you've got big (or saturated) images: you might want to
470downsample them before doing source extraction:
471
472   $ solve-field --downsample 2 ...
473or $ solve-field --downsample 4 ...
474or etc
475
476
477*** When solve-field processes FITS files, it runs them through a
478"sanitizer" which tries to clean up non-standards-compliant images.
479If your FITS files are compliant, this is a waste of time, and you can
480avoid doing it.
481
482   $ solve-field --no-fits2fits ...
483
484
485*** When solve-field processes FITS images, it looks for an existing
486WCS header.  If one is found, it tries to verify that header before
487trying to solve the image blindly.  You can prevent this with:
488
489   $ solve-field --no-verify ...
490
491Note that currently solve-field only understands a small subset of
492valid WCS headers: essentially just the TAN projection with a CD
493matrix (not CROT).
494
495
496*** If you don't want the plots to be produced:
497
498   $ solve-field --no-plots ...
499
500
501*** To convert a list of pixel coordinates to RA,Dec coordinates:
502
503   $ wcs-xy2rd -w wcs-file -i xy-list -o radec-list
504
505Where xy-list is a FITS BINTABLE of the pixel locations of sources;
506recall that FITS specifies that the center of the first pixel is pixel
507coordinate (1,1).
508
509
510*** To convert from RA,Dec to pixels:
511
512   $ wcs-rd2xy -w wcs-file -i radec-list -o xy-list
513
514
515*** To make cool overlay plots: see plotxy, plot-constellations.
516
517
518*** To change the output filenames when processing multiple input
519files: each of the output filename options listed below can include
520"%s", which will be replaced by the base output filename.  (Eg, the
521default for --wcs is "%s.wcs").  If you really want a "%" character in
522your output filename, you have to put "%%".
523
524Outputs include: --new-fits, --kmz, --solved, --cancel, --match,
525--rdls, --corr, --wcs --keep-xylist --pnm
526Also included: --solved-in, --verify
527
528
529*** Reusing files between runs:
530
531The first time you run solve-field, save the source extraction
532results:
533   $ solve-field --keep-xylist %s.xy input.fits ...
534
535On subsequent runs, instead of using the original input file, use the
536saved xylist instead.  Also add --continue to overwrite any output
537file that already exists.
538   $ solve-field input.xy --no-fits2fits --continue ...
539
540To skip previously solved inputs (note that this assumes single-HDU
541inputs):
542   $ solve-field --skip-solved ...
543
544
545Optimizing the code
546-------------------
547
548Here are some things you can do to make the code run faster:
549
550  * we try to guess "-mtune" settings that will work for you; if we're
551    wrong, you can set the environment variable ARCH_FLAGS before
552    compiling:
553
554      $ ARCH_FLAGS="-mtune=nocona" make
555
556    You can find details in the gcc manual:
557      http://gcc.gnu.org/onlinedocs/
558
559    You probably want to look in the section:
560      "GCC Command Options"
561         -> "Hardware Models and Configurations"
562             -> "Intel 386 and AMD x86-64 Options"
563
564    http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/i386-and-x86_002d64-Options.html#i386-and-x86_002d64-Options
565
566
567WHAT ARE ALL THESE PROGRAMS?
568----------------------------
569
570When you "make install", you'll get a bunch of programs in
571/usr/local/astrometry/bin.  Here's a brief synopsis of what each one
572does.  For more details, run the program without arguments (most of
573them give at least a brief summary of what they do).
574
575Image-solving programs
576----------------------
577  * solve-field: main high-level command-line user interface.
578
579  * backend: higher-level solver that reads "augmented xylists";
580called by solve-field.
581
582  * augment-xylist: creates "augmented xylists" from images, which
583include star positions and hints and instructions for solving.
584
585  * blind: low-level command-line solver.
586
587  * image2xy: source extractor.
588
589Plotting programs
590-----------------
591  * plotxy: plots circles, crosses, etc over images.
592
593  * plotquad: draws polygons over images.
594
595  * plot-constellations: annotates images with constellations, bright
596stars, Messier/NGC objects, Henry Draper catalog stars, etc.
597
598  * plotcat: produces density plots given lists of stars.
599
600WCS utilities
601-------------
602
603  * new-wcs: merge a WCS solution with existing FITS header cards; can
604    be used to create a new image file containing the WCS headers.
605
606  * fits-guess-scale: try to guess the scale of an image based on FITS
607    headers.
608
609  * wcsinfo: print simple properties of WCS headers (scale, rotation,
610    etc)
611
612  * wcs-xy2rd, wcs-rd2xy: convert between lists of pixel (x,y) and
613    (RA,Dec) positions.
614
615  * wcs-resample: projects one FITS image onto another image. 
616
617  * wcs-grab/get-wcs: try to interpret an existing WCS header.
618
619Miscellany:
620-----------
621  * an-fitstopnm: converts FITS images into ugly PNM images.
622
623  * get-healpix: which healpix covers a given RA,Dec?
624
625  * hpowned: which small healpixels are inside a big healpixel?
626
627  * control-program: sample code for how you might use Astrometry.net
628    code in your own software.
629
630  * xylist2fits: converts a text list of x,y positions to a FITS
631    binary table.
632
633  * rdlsinfo: print stats about a list of RA,Dec positions (rdlist).
634
635  * xylsinfo: print stats about a list of x,y positions (xylist).
636
637FITS utilities
638--------------
639  * tablist: list values in a FITS binary table.
640
641  * modhead: print or modify FITS header cards.
642
643  * fitscopy: general FITS image / table copier.
644
645  * tabmerge: combines rows in two FITS tables.
646
647  * fitstomatlab: prints out FITS binary tables in a silly format.
648
649  * liststruc: shows the structure of a FITS file.
650
651  * listhead: prints FITS header cards.
652
653  * imcopy: copies FITS images.
654
655  * imarith: does (very) simple arithmetic on FITS images.
656
657  * imstat: computes statistics on FITS images.
658
659  * fitsgetext: pull out individual header or data blocks from
660    multi-HDU FITS files.
661
662  * subtable: pull out a set of columns from a many-column FITS binary
663    table.
664
665  * tabsort: sort a FITS binary table based on values in one column.
666
667  * column-merge: create a FITS binary table that includes columns
668    from two input tables.
669
670  * add-healpix-column: given a FITS binary table containing RA and
671    DEC columns, compute the HEALPIX and add it as a column.
672
673  * resort-xylist: used by solve-field to sort a list of stars using a
674    compromise between background-subtracted and
675    non-background-subtracted flux (because our source extractor
676    sometimes messes up the background subtraction).
677
678  * fits-flip-endian: does endian-swapping of FITS binary tables.
679
680  * fits-dedup: removes duplicate header cards.
681
682Index-building programs
683-----------------------
684
685  * build-index: given a FITS binary table with RA,Dec, build an
686    index.  This is the "easy", recent way.  The old way uses the rest
687    of these programs:
688
689    * usnobtofits, tycho2tofits, nomadtofits, 2masstofits: convert
690      catalogs into FITS binary tables.
691
692    * build-an-catalog: convert input catalogs into a standard FITS
693      binary table format.
694
695    * cut-an: grab a bright, uniform subset of stars from a catalog.
696
697    * startree: build a star kdtree from a catalog.
698
699    * hpquads: find a bright, uniform set of N-star features.
700
701    * codetree: build a kdtree from N-star shape descriptors.
702
703    * unpermute-quads, unpermute-stars: reorder index files for
704      efficiency.
705
706
707
708XYLISTS
709-------
710
711The solve-field program accepts either images or "xylists" (xyls),
712which are just FITS BINTABLE files which contain two columns (float or
713double (E or D) format) which list the pixel coordinates of sources
714(stars, etc) in the image.
715
716To specify the column names (eg, "XIMAGE" and "YIMAGE"):
717  $ solve-field --x-column XIMAGE --y-column YIMAGE ...
718
719Our solver assumes that the sources are listed in order of brightness,
720with the brightest sources first.  If your files aren't sorted, you
721can specify a column by which the file should be sorted.
722  $ solve-field --sort-column FLUX ...
723
724By default it sorts with the largest value first (so it works
725correctly if the column contains FLUX values), but you can reverse
726that by:
727  $ solve-field --sort-ascending --sort-column MAG ...
728
729When using xylists, you should also specify the original width and
730height of the image, in pixels:
731  $ solve-field --width 2000 --height 1500 ...
732
733Alternatively, if the FITS header contains "IMAGEW" and "IMAGEH" keys,
734these will be used.
735
736The solver can deal with multi-extension xylists.  Tis is a convenient
737way to solve a large number of fields at once.  You can tell it which
738extensions it should solve by:
739  $ solve-field --fields 1-100,120,130-200
740
741(Ranges of fields are inclusive, and the first FITS extension is 1, as
742per the FITS standard.)
743
744Unfortunately, the plotting code isn't smart about handling multiple
745fields, so if you're using multi-extension xylists you probably want
746to turn off plotting:
747  $ solve-field --no-plots ...
748
749
750BACKEND CONFIG
751--------------
752
753Because we also operate a web service using most of the same software,
754the local version of the solver is a bit more complicated than it
755really needs to be.  The "solve-field" program takes your input files,
756does source extraction on them to produce an "xylist" -- a FITS
757BINTABLE of source positions -- then takes the information you
758supplied about your fields on the command-line and adds FITS headers
759encoding this information.  We call this file an "augmented xylist";
760we use the filename suffix ".axy".  "solve-field" then calls the
761"backend" program, passing it your axy file.  "backend" reads a config
762file (by default /usr/local/astrometry/etc/backend.cfg) that describes
763things like where to find index files, whether to load all the index
764files at once or run them one at a time, how long to spend on each
765field, and so on.  If you want to force only a certain set of index
766files to load, you can copy the backend.cfg file to a local version
767and change the list of index files that are loaded, and then tell
768solve-field to use this config file:
769
770   $ solve-field --backend-config mybackend.cfg ...
771
772
773SOURCE EXTRACTION USING SEXTRACTOR
774----------------------------------
775
776http://www.astromatic.net/software/sextractor
777
778The "Source Extractor" aka "SExtractor" program by Emmanuel Bertin can
779be used to do source extraction if you don't want to use our own
780bundled "image2xy" program.
781
782NOTE: users have reported that SExtractor 2.4.4 (available in some
783Ubuntu distributions) DOES NOT WORK -- it prints out correct source
784positions as it runs, but the "xyls" output file it produces contains
785all (0,0).  We haven't looked into why this is or how to work around
786it.  Later versions of SExtractor such as 2.8.6 work fine.
787
788You can tell solve-field to use SExtractor like this:
789
790  $ solve-field --use-sextractor ...
791
792By default we use almost all SExtractor's default settings.  The
793exceptions are:
794
795  1) We write a PARAMETERS_NAME file containing:
796         X_IMAGE
797         Y_IMAGE
798         MAG_AUTO
799
800  2) We write a FILTER_NAME file containing a Gaussian PSF with FWHM
801     of 2 pixels.  (See blind/augment-xylist.c "filterstr" for the
802     exact string.)
803
804  3) We set CATALOG_TYPE FITS_1.0
805
806  4) We set CATALOG_NAME to a temp filename.
807
808
809If you want to override any of the settings we use, you can use:
810
811  $ solve-field --use-sextractor --sextractor-config <sex.conf>
812
813In order to reproduce the default behavior, you must:
814
815  1) Create a parameters file like the one we make, and set
816     PARAMETERS_NAME to its filename
817
818  2) Set:
819  $ solve-field --x-column X_IMAGE --y-column Y_IMAGE \
820       --sort-column MAG_AUTO --sort-ascending
821
822  3) Create a filter file like the one we make, and set FILTER_NAME to
823     its filename
824
825
826Note that you can tell solve-field where to find SExtractor with
827
828  $ solve-field --use-sextractor --sextractor-path <path-to-sex-executable>
829
830
831
832WORKAROUNDS
833-----------
834
835*** No python
836
837There are two places we use python: handling images, and filtering
838FITS files.
839
840You can avoid the image-handling code by doing source extraction
841yourself; see the "No netpbm" section below.
842
843You can avoid filtering FITS files by using the "--no-fits2fits"
844option to solve-field.
845
846*** No netpbm
847
848We use the netpbm tools (jpegtopnm, pnmtofits, etc) to convert from
849all sorts of image formats to PNM and FITS.
850
851If you don't have these programs installed, you must do source
852extraction yourself and use "xylists" rather than images as the input
853to solve-field.  See SEXTRACTOR and XYLIST sections above.
854
855ERROR MESSAGES during compiling
856-------------------------------
857
8581.    /bin/sh: line 1: /dev/null: No such file or directory
859
860We've seen this happen on Macs a couple of times.  Reboot and it goes
861away...
862
8632.    makefile.deps:40: deps: No such file or directory
864
865Not a problem.  We use automatic dependency tracking: "make" keeps
866track of which source files depend on which other source files.  These
867dependencies get stored in a file named "deps"; when it doesn't exist,
868"make" tries to rebuild it, but not before printing this message.
869
8703.    os-features-test.c: In function 'main':
871      os-features-test.c:23: warning: implicit declaration of function 'canonicalize_file_name'
872      os-features-test.c:23: warning: initialization makes pointer from integer without a cast
873      /usr/bin/ld: Undefined symbols:
874      _canonicalize_file_name
875      collect2: ld returned 1 exit status
876
877Not a problem.  We provide replacements for a couple of OS-specific
878functions, but we need to decide whether to use them or not.  We do
879that by trying to build a test program and checking whether it works.
880This failure tells us your OS doesn't provide the
881canonicalize_file_name() function, so we plug in a replacement.
882
8834.    configure: WARNING: cfitsio: == No acceptable f77 found in $PATH
884      configure: WARNING: cfitsio: == Cfitsio will be built without Fortran wrapper support
885      drvrfile.c: In function 'file_truncate':
886      drvrfile.c:360: warning: implicit declaration of function 'ftruncate'
887      drvrnet.c: In function 'http_open':
888      drvrnet.c:300: warning: implicit declaration of function 'alarm'
889      drvrnet.c: In function 'http_open_network':
890      drvrnet.c:810: warning: implicit declaration of function 'close'
891      drvrsmem.c: In function 'shared_cleanup':
892      drvrsmem.c:154: warning: implicit declaration of function 'close'
893      group.c: In function 'fits_get_cwd':
894      group.c:5439: warning: implicit declaration of function 'getcwd'
895      ar: creating archive libcfitsio.a
896
897Not a problem; these errors come from cfitsio and we just haven't
898fixed them.
899
900
901LICENSE
902-------
903
904The Astrometry.net code suite is free software licensed under the GNU
905GPL, version 2.  See the file LICENSE for the full terms of the GNU
906GPL.
907
908The index files come with their own license conditions.  See the file
909GETTING-INDEXES for details.
910
911CONTACT
912-------
913
914You can post questions (or maybe even find the answer to your
915questions) at http://forum.astrometry.net .  However, please also send
916an email to "code2 at astrometry dot net" pointing out your post to
917the forum -- we never remember to check the forum!  We would also be
918happy to hear via email any bug reports, comments, critiques, feature
919requests, and in general any reports on your experiences, good or bad.
920
Note: See TracBrowser for help on using the repository browser.