| 1 | Astrometry.net code release |
|---|
| 2 | =========================== |
|---|
| 3 | |
|---|
| 4 | Copyright 2006-2009 Michael Blanton, David W. Hogg, Dustin Lang, Keir |
|---|
| 5 | Mierle and Sam Roweis. |
|---|
| 6 | |
|---|
| 7 | This code is meant as an accompanyment for an upcoming paper and its |
|---|
| 8 | main purpose is to allow reproduction of the results in that paper. |
|---|
| 9 | |
|---|
| 10 | This is research code only and is not intended for scientific use in |
|---|
| 11 | reducing data or other analyses. |
|---|
| 12 | |
|---|
| 13 | This release includes a snapshot of all of the components of our |
|---|
| 14 | current research code, including routines to: |
|---|
| 15 | * Convert raw USNO-B and Tycho2 into FITS format for easier use |
|---|
| 16 | * Uniformize, deduplicate, and cut the FITSified catalogs |
|---|
| 17 | * Build indexes from these cuts |
|---|
| 18 | * Solve the astrometry of images using these indexes |
|---|
| 19 | |
|---|
| 20 | The code includes: |
|---|
| 21 | * A simple but powerful HEALPIX implementation |
|---|
| 22 | * The QFITS library with several modifications |
|---|
| 23 | * libkd, a compact and high-performance kdtree library |
|---|
| 24 | |
|---|
| 25 | In principle, the code in the tarball is sufficient to reproduce our |
|---|
| 26 | entire astrometry system and replicate our blind astrometry results |
|---|
| 27 | (e.g. on the SDSS fields). However, this requires the entire USNO-B |
|---|
| 28 | 1.0 catalogue which is about 80GB in size. For ease of use, we are |
|---|
| 29 | distributing our pre-cooked indexes (about 25 GB total), which are the |
|---|
| 30 | same files we're using for our web service. |
|---|
| 31 | |
|---|
| 32 | INSTALLING |
|---|
| 33 | ---------- |
|---|
| 34 | |
|---|
| 35 | To compile everything and run with full functionality, you will need: |
|---|
| 36 | * GNU build tools (gcc, make, etc.) |
|---|
| 37 | * cairo |
|---|
| 38 | * netpbm |
|---|
| 39 | * libpng |
|---|
| 40 | * libjpeg |
|---|
| 41 | * libz |
|---|
| 42 | * python (probably >= 2.4) |
|---|
| 43 | * numpy |
|---|
| 44 | |
|---|
| 45 | -> On Ubuntu or Debian-like systems, the following should suffice: |
|---|
| 46 | |
|---|
| 47 | $ sudo apt-get install libcairo2-dev libnetpbm10-dev netpbm \ |
|---|
| 48 | libpng12-dev libjpeg-dev python-numpy \ |
|---|
| 49 | zlib1g-dev |
|---|
| 50 | |
|---|
| 51 | -> On RHEL, you'll need something like: |
|---|
| 52 | $ sudo yum install cairo.x86_64 cairo-devel.x86_64 netpbm.x86_64 \ |
|---|
| 53 | netpbm-devel.x86_64 fontconfig-devel.x86_64 \ |
|---|
| 54 | libXrender-devel.x86_64 xorg-x11-proto-devel.x86_64 \ |
|---|
| 55 | zlib-devel |
|---|
| 56 | |
|---|
| 57 | -> On Mac OS X, possibly the easiest way is to install Fink |
|---|
| 58 | (www.finkproject.org), and then use apt-get install as per the |
|---|
| 59 | Debian instructions above (leaving out "zlib1g-dev" because it's |
|---|
| 60 | already included with OSX). Note that to use Fink you will need to |
|---|
| 61 | add something like this in your ~/.profile or ~/.bashrc file: |
|---|
| 62 | |
|---|
| 63 | . /sw/bin/init.sh |
|---|
| 64 | export CFLAGS="-I/usr/local/include -I/sw/include" |
|---|
| 65 | export LDFLAGS="-L/usr/local/lib -L/sw/lib" |
|---|
| 66 | |
|---|
| 67 | |
|---|
| 68 | If you don't have and can't get these libraries, you should still be |
|---|
| 69 | able to compile and use the core parts of the solver, but you will |
|---|
| 70 | miss out on some eye-candy. |
|---|
| 71 | |
|---|
| 72 | Grab the code: |
|---|
| 73 | $ wget http://astrometry.net/downloads/astrometry.net-0.25.tar.bz2 |
|---|
| 74 | $ tar xjf astrometry.net-0.25.tar.bz2 |
|---|
| 75 | $ cd astrometry.net-0.25 |
|---|
| 76 | |
|---|
| 77 | Build the solving system: |
|---|
| 78 | $ make |
|---|
| 79 | |
|---|
| 80 | If you installed the libraries listed above, build the plotting code: |
|---|
| 81 | $ make extra |
|---|
| 82 | |
|---|
| 83 | Install it: |
|---|
| 84 | $ make install |
|---|
| 85 | |
|---|
| 86 | You might see some error message during compilation; see the section |
|---|
| 87 | ERROR MESSAGES below for fixes to common problems. |
|---|
| 88 | |
|---|
| 89 | By default it will be installed in /usr/local/astrometry . |
|---|
| 90 | You can override this by either: |
|---|
| 91 | * editing the top-level Makefile (look for INSTALL_DIR) |
|---|
| 92 | or * defining INSTALL_DIR on the command-line: |
|---|
| 93 | For bash shell: |
|---|
| 94 | $ export INSTALL_DIR=/path/to/astrometry |
|---|
| 95 | $ make install |
|---|
| 96 | or |
|---|
| 97 | $ INSTALL_DIR=/path/to/astrometry make install |
|---|
| 98 | For tcsh shell: |
|---|
| 99 | $ setenv INSTALL_DIR /path/to/astrometry |
|---|
| 100 | $ make install |
|---|
| 101 | |
|---|
| 102 | The astrometry solver is composed of several executables. You may |
|---|
| 103 | want to add the INSTALL_DIR/bin directory to your path: |
|---|
| 104 | For bash shell: |
|---|
| 105 | $ export PATH="$PATH:/usr/local/astrometry/bin" |
|---|
| 106 | For tcsh shell: |
|---|
| 107 | $ setenv PATH "$PATH:/usr/local/astrometry/bin" |
|---|
| 108 | |
|---|
| 109 | |
|---|
| 110 | GETTING INDEXES |
|---|
| 111 | --------------- |
|---|
| 112 | Next, grab some pre-cooked indexes. See the file GETTING-INDEXES for |
|---|
| 113 | details about how to do this. |
|---|
| 114 | |
|---|
| 115 | (If you have indexes from a previous version of the Astrometry.net |
|---|
| 116 | software, see the "Upgrading" section below.) |
|---|
| 117 | |
|---|
| 118 | Each index is designed to solve images within a narrow range of |
|---|
| 119 | scales. The indexes designed to solve small (angular size) images are |
|---|
| 120 | rather large files, so you probably only want to grab the indexes |
|---|
| 121 | required for the images you wish to solve. If you grab extra indexes, |
|---|
| 122 | the solver will run more slowly, but the results should be the same. |
|---|
| 123 | |
|---|
| 124 | Each index contains a large number of "skymarks" (landmarks for the |
|---|
| 125 | sky) that allow our solver to identify your images. The skymarks |
|---|
| 126 | contained in each index have sizes (diameters) within a narrow range. |
|---|
| 127 | You probably want to download indexes whose quads are, say, 10% to |
|---|
| 128 | 100% of the sizes of the images you want to solve. |
|---|
| 129 | |
|---|
| 130 | For example, let's say you have some 1-degree square images. You |
|---|
| 131 | should grab indexes that contain skymarks of size 0.1 to 1 degree, or |
|---|
| 132 | 6 to 60 arcminutes. Referring to the table below, you should grab |
|---|
| 133 | indexes 203 through 209. You might find that the same number of |
|---|
| 134 | fields solve, and faster, using just one or two of the indexes in the |
|---|
| 135 | middle of that range - in our example you might try 205, 206 and 207. |
|---|
| 136 | |
|---|
| 137 | For reference, we used index 202 alone for our SDSS tests (13x9 arcmin |
|---|
| 138 | fields). |
|---|
| 139 | |
|---|
| 140 | --------------------------------------------------------------- |
|---|
| 141 | Index Filename Range of skymark diameters File Size |
|---|
| 142 | (arcminutes) (MB) |
|---|
| 143 | --------------------------------------------------------------- |
|---|
| 144 | index-210+.tar.bz2 60 - 2000 29 |
|---|
| 145 | index-209.tar.bz2 42 - 60 30 |
|---|
| 146 | index-208.tar.bz2 30 - 42 61 |
|---|
| 147 | index-207.tar.bz2 22 - 30 125 |
|---|
| 148 | index-206.tar.bz2 16 - 22 255 |
|---|
| 149 | index-205.tar.bz2 11 - 16 526 |
|---|
| 150 | index-204.tar.bz2 8 - 11 1,023 |
|---|
| 151 | index-203.tar.bz2 5.6 - 8.0 2,089 |
|---|
| 152 | index-202.tar.bz2 4.0 - 5.6 4,260 |
|---|
| 153 | index-201.tar.bz2 2.8 - 4.0 5,878 |
|---|
| 154 | index-200.tar.bz2 2.0 - 2.8 10,058 |
|---|
| 155 | -------------------------------------------------------------- |
|---|
| 156 | |
|---|
| 157 | Download the indexes you need and put them in the top-level |
|---|
| 158 | (astrometry-VERSION) source directory. |
|---|
| 159 | |
|---|
| 160 | Install them: |
|---|
| 161 | $ make install-indexes |
|---|
| 162 | |
|---|
| 163 | (Or you can extract them into the INSTALL_DIR/data directory by hand.) |
|---|
| 164 | |
|---|
| 165 | |
|---|
| 166 | Next, you can (optionally) configure the solver by editing the file |
|---|
| 167 | INSTALL_DIR/etc/backend.cfg |
|---|
| 168 | |
|---|
| 169 | |
|---|
| 170 | UPGRADING INDEX FILES |
|---|
| 171 | --------------------- |
|---|
| 172 | |
|---|
| 173 | The format of the index files has changed from the 0.1 and 0.11 |
|---|
| 174 | releases. There is a program for updating them, and there is also a |
|---|
| 175 | makefile target. |
|---|
| 176 | |
|---|
| 177 | $ make upgrade-indexes |
|---|
| 178 | |
|---|
| 179 | If you want to do it by hand, use the program "fix-bb". Run it on |
|---|
| 180 | each of the ckdt.fits and skdt.fits files in your INSTALL_DIR/data |
|---|
| 181 | directory. |
|---|
| 182 | |
|---|
| 183 | |
|---|
| 184 | BIG-ENDIAN MACHINES |
|---|
| 185 | ------------------- |
|---|
| 186 | |
|---|
| 187 | Most CPUs these days are little-endian. If you have an Intel or AMD |
|---|
| 188 | chip, you can skip this section. The most common big-endian CPU in |
|---|
| 189 | recent times is the PowerPC used in Macs. If you have one of these, |
|---|
| 190 | read on. |
|---|
| 191 | |
|---|
| 192 | The index files we are distributing are for little-endian machines. |
|---|
| 193 | For big-endian machines, you must do the following: |
|---|
| 194 | |
|---|
| 195 | cd /usr/local/astrometry/data |
|---|
| 196 | for f in index-*.fits; do |
|---|
| 197 | fits-flip-endian -i $f -o flip-$f -e 1 -s 4 -e 3 -s 4 -e 4 -s 2 -e 5 -s 8 -e 6 -s 2 -e 8 -s 4 -e 9 -s 4 -e 10 -s 8 -e 11 -s 4 |
|---|
| 198 | for e in 0 2 7; do |
|---|
| 199 | modhead flip-$f"[$e]" ENDIAN 01:02:03:04 |
|---|
| 200 | done |
|---|
| 201 | done |
|---|
| 202 | |
|---|
| 203 | assuming "fits-flip-endian" and "modhead" are in your path. The files |
|---|
| 204 | "flip-index-*.fits" will contain the flipped index files. |
|---|
| 205 | |
|---|
| 206 | If that worked, you can swap the flipped ones into place (while |
|---|
| 207 | saving the originals) with: |
|---|
| 208 | |
|---|
| 209 | cd /usr/local/astrometry/data |
|---|
| 210 | mkdir -p orig |
|---|
| 211 | for f in index-*.fits; do |
|---|
| 212 | echo "backing up $f" |
|---|
| 213 | mv -n $f orig/$f |
|---|
| 214 | echo "moving $f into place" |
|---|
| 215 | mv -n flip-$f $f |
|---|
| 216 | done |
|---|
| 217 | |
|---|
| 218 | SOLVING |
|---|
| 219 | ------- |
|---|
| 220 | |
|---|
| 221 | Finally, solve some fields. |
|---|
| 222 | |
|---|
| 223 | (If you didn't build the plotting commands, add "--no-plots" to the |
|---|
| 224 | command lines below.) |
|---|
| 225 | |
|---|
| 226 | If you have any of indexes 213 to 218: |
|---|
| 227 | $ solve-field --scale-low 10 demo/apod4.jpg |
|---|
| 228 | |
|---|
| 229 | If you have index 219: |
|---|
| 230 | $ solve-field --scale-low 30 demo/apod5.jpg |
|---|
| 231 | |
|---|
| 232 | If you have any of indexes 210 to 214: |
|---|
| 233 | $ solve-field --scale-low 1 demo/apod3.jpg |
|---|
| 234 | |
|---|
| 235 | If you have any of indexes 206 to 211: |
|---|
| 236 | $ solve-field --scale-low 1 demo/apod2.jpg |
|---|
| 237 | |
|---|
| 238 | If you have any of indexes 203 to 205: |
|---|
| 239 | $ solve-field apod1.jpg |
|---|
| 240 | |
|---|
| 241 | If you have any of indexes 200 to 203: |
|---|
| 242 | $ solve-field demo/sdss.jpg |
|---|
| 243 | |
|---|
| 244 | |
|---|
| 245 | Copyrights and credits for the demo images are listed in the file demo/CREDITS . |
|---|
| 246 | |
|---|
| 247 | Note that you can also give solve-field a URL rather than a file as input: |
|---|
| 248 | $ solve-field --out apod1b http://antwrp.gsfc.nasa.gov/apod/image/0302/ngc2264_croman_c3.jpg |
|---|
| 249 | |
|---|
| 250 | |
|---|
| 251 | If you don't have the netpbm tools (eg jpegtopnm), do this instead: |
|---|
| 252 | |
|---|
| 253 | If you have any of indexes 213 to 218: |
|---|
| 254 | $ solve-field --scale-low 10 demo/apod4.xyls |
|---|
| 255 | |
|---|
| 256 | If you have index 219: |
|---|
| 257 | $ solve-field --scale-low 30 demo/apod5.xyls |
|---|
| 258 | |
|---|
| 259 | If you have any of indexes 210 to 214: |
|---|
| 260 | $ solve-field --scale-low 1 demo/apod3.xyls |
|---|
| 261 | |
|---|
| 262 | If you have any of indexes 206 to 211: |
|---|
| 263 | $ solve-field --scale-low 1 demo/apod2.xyls |
|---|
| 264 | |
|---|
| 265 | If you have any of indexes 203 to 205: |
|---|
| 266 | $ solve-field demo/apod1.xyls |
|---|
| 267 | |
|---|
| 268 | If you have any of indexes 200 to 203: |
|---|
| 269 | $ solve-field demo/sdss.xyls |
|---|
| 270 | |
|---|
| 271 | |
|---|
| 272 | OUTPUT FILES |
|---|
| 273 | ------------ |
|---|
| 274 | |
|---|
| 275 | <base>-ngc.png : an annotation of the image. |
|---|
| 276 | <base>.wcs : a FITS WCS header for the solution. |
|---|
| 277 | <base>.new : a new FITS file containing the WCS header. |
|---|
| 278 | <base>-objs.png : a plot of the sources (stars) we extracted from |
|---|
| 279 | the image. |
|---|
| 280 | <base>-indx.png : sources (red), plus stars from the index (green), |
|---|
| 281 | plus the skymark ("quad") used to solve the |
|---|
| 282 | image. |
|---|
| 283 | <base>-indx.xyls : a FITS BINTABLE with the pixel locations of |
|---|
| 284 | stars from the index. |
|---|
| 285 | <base>.rdls : a FITS BINTABLE with the RA,Dec of sources we |
|---|
| 286 | extracted from the image. |
|---|
| 287 | <base>.axy : a FITS BINTABLE of the sources we extracted, plus |
|---|
| 288 | headers that describe the job (how the image is |
|---|
| 289 | going to be solved). |
|---|
| 290 | <base>.solved : exists and contains (binary) 1 if the field solved. |
|---|
| 291 | <base>.match : a FITS BINTABLE describing the quad match that |
|---|
| 292 | solved the image. |
|---|
| 293 | <base>.kmz : (optional) KMZ file for Google Sky-in-Earth. You need |
|---|
| 294 | to have "wcs2kml" in your PATH. See |
|---|
| 295 | http://code.google.com/p/wcs2kml/downloads/list |
|---|
| 296 | http://code.google.com/p/google-gflags/downloads/list |
|---|
| 297 | |
|---|
| 298 | |
|---|
| 299 | TRICKS AND TIPS |
|---|
| 300 | --------------- |
|---|
| 301 | |
|---|
| 302 | *** To lower the CPU time limit before giving up: |
|---|
| 303 | |
|---|
| 304 | $ solve-field --cpu-limit 30 ... |
|---|
| 305 | |
|---|
| 306 | will make it give up after 30 seconds. |
|---|
| 307 | |
|---|
| 308 | (Note, however, that the "backend" configuration file (backend.cfg) |
|---|
| 309 | puts a limit on the CPU time that is spent on an image; solve-field |
|---|
| 310 | can reduce this but not increase it.) |
|---|
| 311 | |
|---|
| 312 | |
|---|
| 313 | *** Scale of the image: if you provide bounds (lower and upper limits) |
|---|
| 314 | on the size of the image you are trying to solve, solving can be much |
|---|
| 315 | faster. In the last examples above, for example, we specified that |
|---|
| 316 | the field is at least 30 degrees wide: this means that we don't need |
|---|
| 317 | to search for matches in the indexes that contain only tiny skymarks. |
|---|
| 318 | |
|---|
| 319 | Eg, to specify that the image is between 1 and 2 degrees wide: |
|---|
| 320 | |
|---|
| 321 | $ solve-field --scale-units degwidth --scale-low 1 --scale-high 2 ... |
|---|
| 322 | |
|---|
| 323 | If you know the pixel scale instead: |
|---|
| 324 | |
|---|
| 325 | $ solve-field --scale-units arcsecperpix \ |
|---|
| 326 | --scale-low 0.386 --scale-high 0.406 ... |
|---|
| 327 | |
|---|
| 328 | When you tell solve-field the scale of your image, it uses this to |
|---|
| 329 | decide which indexes to try to use to solve your image; each index |
|---|
| 330 | contains quads whose scale is within a certain range, so if these |
|---|
| 331 | quads are too big or too small to be in your image, there is no need |
|---|
| 332 | to look in that index. It is also used while matching quads: a small |
|---|
| 333 | quad in your image is not allowed to match a large quad in the index |
|---|
| 334 | if such a match would cause the image scale to be outside the bounds |
|---|
| 335 | you specified. However, all these checks are done before computing a |
|---|
| 336 | best-fit WCS solution and polynomial distortion terms, so it is |
|---|
| 337 | possible (though rare) for the final solution to fall outside the |
|---|
| 338 | limits you specified. This should only happen when the solution is |
|---|
| 339 | correct, but you gave incorrect inputs, so you shouldn't be |
|---|
| 340 | complaining! :) |
|---|
| 341 | |
|---|
| 342 | |
|---|
| 343 | *** Guess the scale: solve-field can try to guess your image's scale |
|---|
| 344 | from a number of different FITS header values. When it's right, this |
|---|
| 345 | often speeds up solving a lot, and when it's wrong it doesn't cost |
|---|
| 346 | much. Enable this with: |
|---|
| 347 | |
|---|
| 348 | $ solve-field --guess-scale ... |
|---|
| 349 | |
|---|
| 350 | *** Depth. The solver works by looking at sources in your image, |
|---|
| 351 | starting with the brightest. It searches for all "skymarks" that can |
|---|
| 352 | be built from the N brightest stars before considering star N+1. When |
|---|
| 353 | using several indexes, it can be much faster to search for many |
|---|
| 354 | skymarks in one index before switching to the next one. This flag |
|---|
| 355 | lets you control when the solver switches between indexes. It also |
|---|
| 356 | lets you control how much effort the solver puts in before giving up - |
|---|
| 357 | by default it looks at all the sources in your image, and usually |
|---|
| 358 | times out before this finishes. |
|---|
| 359 | |
|---|
| 360 | Eg, to first look at sources 1-20 in all indexes, then sources 21-30 |
|---|
| 361 | in all indexes, then 31-40: |
|---|
| 362 | |
|---|
| 363 | $ solve-field --depth 20,30,40 ... |
|---|
| 364 | or $ solve-field --depth 1-20 --depth 21-30 --depth 31-40 ... |
|---|
| 365 | |
|---|
| 366 | Sources are numbered starting at one, and ranges are inclusive. If |
|---|
| 367 | you don't give a lower limit, it will take 1 + the previous upper |
|---|
| 368 | limit. To look at a single source, do |
|---|
| 369 | |
|---|
| 370 | $ solve-field --depth 42-42 ... |
|---|
| 371 | |
|---|
| 372 | |
|---|
| 373 | *** Our source extractor sometimes estimates the background badly, so |
|---|
| 374 | by default we sort the stars by brightness using a compromise between |
|---|
| 375 | the raw and background-subtracted flux estimates. For images without |
|---|
| 376 | much nebulosity, you might find that using the background-subtracted |
|---|
| 377 | fluxes yields faster results. Enable this by: |
|---|
| 378 | |
|---|
| 379 | $ solve-field --resort ... |
|---|
| 380 | |
|---|
| 381 | |
|---|
| 382 | *** If you've got big images: you might want to downsample them before |
|---|
| 383 | doing source extraction: |
|---|
| 384 | |
|---|
| 385 | $ solve-field --downsample 2 ... |
|---|
| 386 | or $ solve-field --downsample 4 ... |
|---|
| 387 | or etc |
|---|
| 388 | |
|---|
| 389 | |
|---|
| 390 | *** When solve-field processes FITS files, it runs them through a |
|---|
| 391 | "sanitizer" which tries to clean up non-standards-compliant images. |
|---|
| 392 | If your FITS files are compliant, this is a waste of time, and you can |
|---|
| 393 | avoid doing it. |
|---|
| 394 | |
|---|
| 395 | $ solve-field --no-fits2fits ... |
|---|
| 396 | |
|---|
| 397 | |
|---|
| 398 | *** When solve-field processes FITS images, it looks for an existing |
|---|
| 399 | WCS header. If one is found, it tries to verify that header before |
|---|
| 400 | trying to solve the image blindly. You can prevent this with: |
|---|
| 401 | |
|---|
| 402 | $ solve-field --no-verify ... |
|---|
| 403 | |
|---|
| 404 | Note that currently solve-field only understands a small subset of |
|---|
| 405 | valid WCS headers: essentially just the TAN projection with a CD |
|---|
| 406 | matrix (not CROT). |
|---|
| 407 | |
|---|
| 408 | |
|---|
| 409 | *** If you don't want the plots to be produced: |
|---|
| 410 | |
|---|
| 411 | $ solve-field --no-plots ... |
|---|
| 412 | |
|---|
| 413 | |
|---|
| 414 | *** "I know where my image is to within 1 arcminute, how can I tell |
|---|
| 415 | solve-field to only look there?" |
|---|
| 416 | |
|---|
| 417 | In this release of the software, you can't. We've got plans for v0.3, |
|---|
| 418 | but don't hold your breath! There is one thing you can do: if you |
|---|
| 419 | know that your image is within one HEALPix |
|---|
| 420 | (http://healpix.jpl.nasa.gov/), and if you're using the |
|---|
| 421 | small-angular-scale indexes with filenames like index-###-##.*.fits, |
|---|
| 422 | then each of these indexes covers one HEALPix (the second number is |
|---|
| 423 | the HEALPix number); you can edit the "backend.cfg" config file to |
|---|
| 424 | tell the "backend" program to only load the index that contains your |
|---|
| 425 | image. |
|---|
| 426 | |
|---|
| 427 | |
|---|
| 428 | *** To convert a list of pixel coordinates to RA,Dec coordinates: |
|---|
| 429 | |
|---|
| 430 | $ wcs-xy2rd -w wcs-file -i xy-list -o radec-list |
|---|
| 431 | |
|---|
| 432 | Where xy-list is a FITS BINTABLE of the pixel locations of sources; |
|---|
| 433 | recall that FITS specifies that the center of the first pixel is pixel |
|---|
| 434 | coordinate (1,1). |
|---|
| 435 | |
|---|
| 436 | |
|---|
| 437 | *** To convert from RA,Dec to pixels: |
|---|
| 438 | |
|---|
| 439 | $ wcs-rd2xy -w wcs-file -i radec-list -o xy-list |
|---|
| 440 | |
|---|
| 441 | |
|---|
| 442 | *** To make cool overlay plots: see plotxy, plot-constellations. |
|---|
| 443 | |
|---|
| 444 | |
|---|
| 445 | *** To change the output filenames when processing multiple input |
|---|
| 446 | files: each of the output filename options listed below can include |
|---|
| 447 | "%s", which will be replaced by the base output filename. (Eg, the |
|---|
| 448 | default for --wcs is "%s.wcs"). If you really want a "%" character in |
|---|
| 449 | your output filename, you have to put "%%". |
|---|
| 450 | |
|---|
| 451 | Outputs include: --new-fits, --kmz, --solved, --cancel, --match, |
|---|
| 452 | --rdls, --corr, --wcs --keep-xylist --pnm |
|---|
| 453 | Also included: --solved-in, --verify |
|---|
| 454 | |
|---|
| 455 | |
|---|
| 456 | *** Reusing files between runs: |
|---|
| 457 | |
|---|
| 458 | The first time you run solve-field, save the source extraction |
|---|
| 459 | results: |
|---|
| 460 | $ solve-field --keep-xylist %s.xy input.fits ... |
|---|
| 461 | |
|---|
| 462 | On subsequent runs, instead of using the original input file, use the |
|---|
| 463 | saved xylist instead. Also add --continue to overwrite any output |
|---|
| 464 | file that already exists. |
|---|
| 465 | $ solve-field input.xy --no-fits2fits --continue ... |
|---|
| 466 | |
|---|
| 467 | To skip previously solved inputs (note that this assumes single-HDU |
|---|
| 468 | inputs): |
|---|
| 469 | $ solve-field --skip-solved ... |
|---|
| 470 | |
|---|
| 471 | |
|---|
| 472 | Optimizing the code |
|---|
| 473 | ------------------- |
|---|
| 474 | |
|---|
| 475 | Here are some things you can do to make the code run faster: |
|---|
| 476 | |
|---|
| 477 | * we try to guess "-mtune" settings that will work for you; if we're |
|---|
| 478 | wrong, you can set the environment variable ARCH_FLAGS before |
|---|
| 479 | compiling: |
|---|
| 480 | |
|---|
| 481 | $ ARCH_FLAGS="-mtune=nocona" make |
|---|
| 482 | |
|---|
| 483 | You can find details in the gcc manual: |
|---|
| 484 | http://gcc.gnu.org/onlinedocs/ |
|---|
| 485 | |
|---|
| 486 | You probably want to look in the section: |
|---|
| 487 | "GCC Command Options" |
|---|
| 488 | -> "Hardware Models and Configurations" |
|---|
| 489 | -> "Intel 386 and AMD x86-64 Options" |
|---|
| 490 | |
|---|
| 491 | http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/i386-and-x86_002d64-Options.html#i386-and-x86_002d64-Options |
|---|
| 492 | |
|---|
| 493 | |
|---|
| 494 | WHAT ARE ALL THESE PROGRAMS? |
|---|
| 495 | ---------------------------- |
|---|
| 496 | |
|---|
| 497 | When you "make install", you'll get a bunch of programs in |
|---|
| 498 | /usr/local/astrometry/bin. Here's a brief synopsis of what each one |
|---|
| 499 | does. For more details, run the program without arguments (most of |
|---|
| 500 | them give at least a brief summary of what they do). |
|---|
| 501 | |
|---|
| 502 | Image-solving programs |
|---|
| 503 | ---------------------- |
|---|
| 504 | * solve-field: main high-level command-line user interface. |
|---|
| 505 | |
|---|
| 506 | * backend: higher-level solver that reads "augmented xylists"; |
|---|
| 507 | called by solve-field. |
|---|
| 508 | |
|---|
| 509 | * augment-xylist: creates "augmented xylists" from images, which |
|---|
| 510 | include star positions and hints and instructions for solving. |
|---|
| 511 | |
|---|
| 512 | * blind: low-level command-line solver. |
|---|
| 513 | |
|---|
| 514 | * image2xy: source extractor. |
|---|
| 515 | |
|---|
| 516 | Plotting programs |
|---|
| 517 | ----------------- |
|---|
| 518 | * plotxy: plots circles, crosses, etc over images. |
|---|
| 519 | |
|---|
| 520 | * plotquad: draws polygons over images. |
|---|
| 521 | |
|---|
| 522 | * plot-constellations: annotates images with constellations, bright |
|---|
| 523 | stars, Messier/NGC objects, Henry Draper catalog stars, etc. |
|---|
| 524 | |
|---|
| 525 | * plotcat: produces density plots given lists of stars. |
|---|
| 526 | |
|---|
| 527 | WCS utilities |
|---|
| 528 | ------------- |
|---|
| 529 | * new-wcs: merge a WCS solution with existing FITS header cards; can |
|---|
| 530 | be used to create a new image file containing the WCS headers. |
|---|
| 531 | |
|---|
| 532 | * fits-guess-scale: try to guess the scale of an image based on FITS |
|---|
| 533 | headers. |
|---|
| 534 | |
|---|
| 535 | * wcsinfo: print simple properties of WCS headers (scale, rotation, etc) |
|---|
| 536 | |
|---|
| 537 | * wcs-xy2rd, wcs-rd2xy: convert between lists of pixel (x,y) and |
|---|
| 538 | (RA,Dec) positions. |
|---|
| 539 | |
|---|
| 540 | * wcs-resample: projects one FITS image onto another image. |
|---|
| 541 | |
|---|
| 542 | * wcs-grab/get-wcs: try to interpret an existing WCS header. |
|---|
| 543 | |
|---|
| 544 | Miscellany: |
|---|
| 545 | ----------- |
|---|
| 546 | * an-fitstopnm: converts FITS images into ugly PNM images. |
|---|
| 547 | |
|---|
| 548 | * get-healpix: which healpix covers a given RA,Dec? |
|---|
| 549 | |
|---|
| 550 | * hpowned: which small healpixels are inside a big healpixel? |
|---|
| 551 | |
|---|
| 552 | * control-program: sample code for how you might use astrometry.net |
|---|
| 553 | code in your own software. |
|---|
| 554 | |
|---|
| 555 | * xylist2fits: converts a text list of x,y positions to a FITS |
|---|
| 556 | binary table. |
|---|
| 557 | |
|---|
| 558 | * rdlsinfo: print stats about a list of RA,Dec positions (rdlist). |
|---|
| 559 | |
|---|
| 560 | * xylsinfo: print stats about a list of x,y positions (xylist). |
|---|
| 561 | |
|---|
| 562 | FITS utilities |
|---|
| 563 | -------------- |
|---|
| 564 | * tablist: list values in a FITS binary table. |
|---|
| 565 | |
|---|
| 566 | * modhead: print or modify FITS header cards. |
|---|
| 567 | |
|---|
| 568 | * fitscopy: general FITS image / table copier. |
|---|
| 569 | |
|---|
| 570 | * tabmerge: combines rows in two FITS tables. |
|---|
| 571 | |
|---|
| 572 | * fitstomatlab: prints out FITS binary tables in a silly format. |
|---|
| 573 | |
|---|
| 574 | * liststruc: shows the structure of a FITS file. |
|---|
| 575 | |
|---|
| 576 | * listhead: prints FITS header cards. |
|---|
| 577 | |
|---|
| 578 | * imcopy: copies FITS images. |
|---|
| 579 | |
|---|
| 580 | * imarith: does (very) simple arithmetic on FITS images. |
|---|
| 581 | |
|---|
| 582 | * imstat: computes statistics on FITS images. |
|---|
| 583 | |
|---|
| 584 | * fitsgetext: pull out individual header or data blocks from |
|---|
| 585 | multi-HDU FITS files. |
|---|
| 586 | |
|---|
| 587 | * subtable: pull out a set of columns from a many-column FITS binary |
|---|
| 588 | table. |
|---|
| 589 | |
|---|
| 590 | * tabsort: sort a FITS binary table based on values in one column. |
|---|
| 591 | |
|---|
| 592 | * column-merge: create a FITS binary table that includes columns |
|---|
| 593 | from two input tables. |
|---|
| 594 | |
|---|
| 595 | * add-healpix-column: given a FITS binary table containing RA and |
|---|
| 596 | DEC columns, compute the HEALPIX and add it as a column. |
|---|
| 597 | |
|---|
| 598 | * resort-xylist: used by solve-field to sort a list of stars using a |
|---|
| 599 | compromise between background-subtracted and non-background-subtracted |
|---|
| 600 | flux (because our source extractor sometimes messes up the background |
|---|
| 601 | subtraction). |
|---|
| 602 | |
|---|
| 603 | * fits-flip-endian: does endian-swapping of FITS binary tables. |
|---|
| 604 | |
|---|
| 605 | * fits-dedup: removes duplicate header cards. |
|---|
| 606 | |
|---|
| 607 | Index-building programs |
|---|
| 608 | ----------------------- |
|---|
| 609 | * usnobtofits, tycho2tofits, nomadtofits, 2masstofits: convert |
|---|
| 610 | catalogs into FITS binary tables. |
|---|
| 611 | |
|---|
| 612 | * build-an-catalog: convert input catalogs into a standard FITS |
|---|
| 613 | binary table format. |
|---|
| 614 | |
|---|
| 615 | * cut-an: grab a bright, uniform subset of stars from a catalog. |
|---|
| 616 | |
|---|
| 617 | * startree: build a star kdtree from a catalog. |
|---|
| 618 | |
|---|
| 619 | * hpquads: find a bright, uniform set of N-star features. |
|---|
| 620 | |
|---|
| 621 | * codetree: build a kdtree from N-star shape descriptors. |
|---|
| 622 | |
|---|
| 623 | * unpermute-quads, unpermute-stars: reorder index files for |
|---|
| 624 | efficiency. |
|---|
| 625 | |
|---|
| 626 | |
|---|
| 627 | |
|---|
| 628 | XYLISTS |
|---|
| 629 | ------- |
|---|
| 630 | |
|---|
| 631 | The solve-field program accepts either images or "xylists" (xyls), |
|---|
| 632 | which are just FITS BINTABLE files which contain two columns (float or |
|---|
| 633 | double (E or D) format) which list the pixel coordinates of sources |
|---|
| 634 | (stars, etc) in the image. |
|---|
| 635 | |
|---|
| 636 | To specify the column names (eg, "XIMAGE" and "YIMAGE"): |
|---|
| 637 | $ solve-field --x-column XIMAGE --y-column YIMAGE ... |
|---|
| 638 | |
|---|
| 639 | Our solver assumes that the sources are listed in order of brightness, |
|---|
| 640 | with the brightest sources first. If your files aren't sorted, you |
|---|
| 641 | can specify a column by which the file should be sorted. |
|---|
| 642 | $ solve-field --sort-column FLUX ... |
|---|
| 643 | |
|---|
| 644 | By default it sorts with the largest value first (so it works |
|---|
| 645 | correctly if the column contains FLUX values), but you can reverse |
|---|
| 646 | that by: |
|---|
| 647 | $ solve-field --sort-ascending --sort-column MAG ... |
|---|
| 648 | |
|---|
| 649 | When using xylists, you should also specify the original width and |
|---|
| 650 | height of the image, in pixels: |
|---|
| 651 | $ solve-field --width 2000 --height 1500 ... |
|---|
| 652 | |
|---|
| 653 | Alternatively, if the FITS header contains "IMAGEW" and "IMAGEH" keys, |
|---|
| 654 | these will be used. |
|---|
| 655 | |
|---|
| 656 | The solver can deal with multi-extension xylists; indeed, this is a |
|---|
| 657 | convenient way to solve a large number of fields at once. You can |
|---|
| 658 | tell it which extensions it should solve by: |
|---|
| 659 | $ solve-field --fields 1-100,120,130-200 |
|---|
| 660 | |
|---|
| 661 | (Ranges of fields are inclusive, and the first FITS extension is 1, as |
|---|
| 662 | per the FITS standard.) |
|---|
| 663 | |
|---|
| 664 | Unfortunately, the plotting code isn't smart about handling multiple |
|---|
| 665 | fields, so if you're using multi-extension xylists you probably want |
|---|
| 666 | to turn off plotting: |
|---|
| 667 | $ solve-field --no-plots ... |
|---|
| 668 | |
|---|
| 669 | |
|---|
| 670 | BACKEND CONFIG |
|---|
| 671 | -------------- |
|---|
| 672 | Because we also operate a web service using most of the same software, |
|---|
| 673 | the local version of the solver is a bit more complicated than it |
|---|
| 674 | really needs to be. The "solve-field" program takes your input files, |
|---|
| 675 | does source extraction on them to produce an "xylist" -- a FITS |
|---|
| 676 | BINTABLE of source positions -- then takes the information you |
|---|
| 677 | supplied about your fields on the command-line and adds FITS headers |
|---|
| 678 | encoding this information. We call this file an "augmented xylist"; |
|---|
| 679 | we use the filename suffix ".axy". "solve-field" then calls the |
|---|
| 680 | "backend" program, passing it your axy file. "backend" reads a config |
|---|
| 681 | file (by default /usr/local/astrometry/etc/backend.cfg) that describes |
|---|
| 682 | things like where to find index files, whether to load all the indexes |
|---|
| 683 | at once or run them one at a time, how long to spend on each field, |
|---|
| 684 | and so on. If you want to force only a certain set of indexes to |
|---|
| 685 | load, you can copy the backend.cfg file to a local version and change |
|---|
| 686 | the list of indexes that are loaded, and then tell solve-field to use |
|---|
| 687 | this config file: |
|---|
| 688 | |
|---|
| 689 | $ solve-field --backend-config mybackend.cfg ... |
|---|
| 690 | |
|---|
| 691 | |
|---|
| 692 | SOURCE EXTRACTION USING SEXTRACTOR |
|---|
| 693 | ---------------------------------- |
|---|
| 694 | http://terapix.iap.fr/rubrique.php?id_rubrique=91/ |
|---|
| 695 | |
|---|
| 696 | The "Source Extractor" aka "SExtractor" program by Emmanuel Bertin can |
|---|
| 697 | be used to do source extraction if you don't want to use our own |
|---|
| 698 | bundled "image2xy" program. |
|---|
| 699 | |
|---|
| 700 | You can tell solve-field to use SExtractor like this: |
|---|
| 701 | |
|---|
| 702 | $ solve-field --use-sextractor ... |
|---|
| 703 | |
|---|
| 704 | By default we use almost all SExtractor's default settings. The |
|---|
| 705 | exceptions are: |
|---|
| 706 | |
|---|
| 707 | 1) We write a PARAMETERS_NAME file containing: |
|---|
| 708 | X_IMAGE |
|---|
| 709 | Y_IMAGE |
|---|
| 710 | MAG_AUTO |
|---|
| 711 | |
|---|
| 712 | 2) We write a FILTER_NAME file containing a Gaussian PSF with FWHM |
|---|
| 713 | of 2 pixels. (See blind/augment-xylist.c "filterstr" for the |
|---|
| 714 | exact string.) |
|---|
| 715 | |
|---|
| 716 | 3) We set CATALOG_TYPE FITS_1.0 |
|---|
| 717 | |
|---|
| 718 | 4) We set CATALOG_NAME to a temp filename. |
|---|
| 719 | |
|---|
| 720 | |
|---|
| 721 | If you want to override any of the settings we use, you can use: |
|---|
| 722 | |
|---|
| 723 | $ solve-field --use-sextractor --sextractor-config <sex.conf> |
|---|
| 724 | |
|---|
| 725 | In order to reproduce the default behavior, you must: |
|---|
| 726 | |
|---|
| 727 | 1) Create a parameters file like the one we make, and set |
|---|
| 728 | PARAMETERS_NAME to its filename |
|---|
| 729 | |
|---|
| 730 | 2) Set: |
|---|
| 731 | $ solve-field --x-column X_IMAGE --y-column Y_IMAGE \ |
|---|
| 732 | --sort-column MAG_AUTO --sort-ascending |
|---|
| 733 | |
|---|
| 734 | 3) Create a filter file like the one we make, and set FILTER_NAME to |
|---|
| 735 | its filename |
|---|
| 736 | |
|---|
| 737 | |
|---|
| 738 | Note that you can tell solve-field where to find SExtractor with |
|---|
| 739 | |
|---|
| 740 | $ solve-field --use-sextractor --sextractor-path <path-to-sex-executable> |
|---|
| 741 | |
|---|
| 742 | |
|---|
| 743 | |
|---|
| 744 | WORKAROUNDS |
|---|
| 745 | ----------- |
|---|
| 746 | *** No python |
|---|
| 747 | |
|---|
| 748 | There are two places we use python: handling images, and filtering FITS files. |
|---|
| 749 | |
|---|
| 750 | You can avoid the image-handling code by doing source extraction |
|---|
| 751 | yourself; see the "No netpbm" section below. |
|---|
| 752 | |
|---|
| 753 | You can avoid filtering FITS files by using the "--no-fits2fits" |
|---|
| 754 | option to solve-field. |
|---|
| 755 | |
|---|
| 756 | *** No netpbm |
|---|
| 757 | |
|---|
| 758 | We use the netpbm tools (jpegtopnm, pnmtofits, etc) to convert from |
|---|
| 759 | all sorts of image formats to PNM and FITS. |
|---|
| 760 | |
|---|
| 761 | If you don't have these programs installed, you must do source |
|---|
| 762 | extraction yourself and use "xylists" rather than images as the input |
|---|
| 763 | to solve-field. See SEXTRACTOR and XYLIST sections above. |
|---|
| 764 | |
|---|
| 765 | ERROR MESSAGES during compiling |
|---|
| 766 | ------------------------------- |
|---|
| 767 | |
|---|
| 768 | 1. /bin/sh: line 1: /dev/null: No such file or directory |
|---|
| 769 | |
|---|
| 770 | We've seen this happen on Macs a couple of times. Reboot and it goes |
|---|
| 771 | away... |
|---|
| 772 | |
|---|
| 773 | 2. makefile.deps:40: deps: No such file or directory |
|---|
| 774 | |
|---|
| 775 | Not a problem. We use automatic dependency tracking: "make" keeps |
|---|
| 776 | track of which source files depend on which other source files. These |
|---|
| 777 | dependencies get stored in a file named "deps"; when it doesn't exist, |
|---|
| 778 | "make" tries to rebuild it, but not before printing this message. |
|---|
| 779 | |
|---|
| 780 | 3. gnu-specific-test.c: In function 'main': |
|---|
| 781 | gnu-specific-test.c:23: warning: implicit declaration of function 'canonicalize_file_name' |
|---|
| 782 | gnu-specific-test.c:23: warning: initialization makes pointer from integer without a cast |
|---|
| 783 | /usr/bin/ld: Undefined symbols: |
|---|
| 784 | _canonicalize_file_name |
|---|
| 785 | collect2: ld returned 1 exit status |
|---|
| 786 | |
|---|
| 787 | Not a problem. We provide replacements for a couple of GNU-specific |
|---|
| 788 | functions, but we need to decide whether to use them or not. We do |
|---|
| 789 | that by trying to build a test program and checking whether it works. |
|---|
| 790 | This failure tells us your OS doesn't provide the |
|---|
| 791 | canonicalize_file_name() function, so we plug in a replacement. |
|---|
| 792 | |
|---|
| 793 | 4. configure: WARNING: cfitsio: == No acceptable f77 found in $PATH |
|---|
| 794 | configure: WARNING: cfitsio: == Cfitsio will be built without Fortran wrapper support |
|---|
| 795 | drvrfile.c: In function 'file_truncate': |
|---|
| 796 | drvrfile.c:360: warning: implicit declaration of function 'ftruncate' |
|---|
| 797 | drvrnet.c: In function 'http_open': |
|---|
| 798 | drvrnet.c:300: warning: implicit declaration of function 'alarm' |
|---|
| 799 | drvrnet.c: In function 'http_open_network': |
|---|
| 800 | drvrnet.c:810: warning: implicit declaration of function 'close' |
|---|
| 801 | drvrsmem.c: In function 'shared_cleanup': |
|---|
| 802 | drvrsmem.c:154: warning: implicit declaration of function 'close' |
|---|
| 803 | group.c: In function 'fits_get_cwd': |
|---|
| 804 | group.c:5439: warning: implicit declaration of function 'getcwd' |
|---|
| 805 | ar: creating archive libcfitsio.a |
|---|
| 806 | |
|---|
| 807 | Not a problem; these errors come from cfitsio and we just haven't |
|---|
| 808 | fixed them. |
|---|
| 809 | |
|---|
| 810 | |
|---|
| 811 | LICENSE |
|---|
| 812 | ------- |
|---|
| 813 | |
|---|
| 814 | The Astrometry.net code suite is free software licensed under the GNU |
|---|
| 815 | GPL, version 2. See the file LICENSE for the full terms of the GNU |
|---|
| 816 | GPL. |
|---|
| 817 | |
|---|
| 818 | The index files come with their own license conditions. See the file |
|---|
| 819 | GETTING-INDEXES for details. |
|---|
| 820 | |
|---|
| 821 | CONTACT |
|---|
| 822 | ------- |
|---|
| 823 | Please send inquires and bug reports to the email address "code" at |
|---|
| 824 | the domain "astrometry.net". |
|---|
| 825 | |
|---|