CSLab Development

Data

Some of the astronomical data is in /p/learning/stars/:

  • /p/learning/stars/CATALOGUES - Raw binary catalogs for GALEX and a couple cuts of USNO
  • /p/learning/stars/GALEX_FIELDS - The fields GALEX sent us as a challenge
  • /p/learning/stars/SDSS_FIELDS - SDSS fields in xyls and rdls format
  • /p/learning/stars/INDEXES/ - indexes

There are also directories stars2 through stars10 that are links to additional disk partitions.

We also have some new RAIDed disk space with reasonable partition sizes. I suggest you create the following links: 

cd
ln -s /w/284/stars284/ raid1
ln -s /w/285/stars285/ raid2
ln -s /w/286/stars286/ raid3

Some of the workstations have large local disks that are not backed up. They're called /nobackup* and can be used for working with temporary data. But most of the stuff we're interested in just goes on the RAID space.

Machines

You'll want to use one of the Linux machines, not the Solaris ones. Run: 

uname

if you get Linux, you're happy. If you get SunOS or Solaris, panic and flee.

You can use the general-access Linux machine age.cs, but don't run big compute jobs there. There are compute servers: cluster0.ai through cluster23.ai. You can also use the workstations used by our group members: monte.ai and axon.ai (dstn), gauss.ai (sam).

Groups

groups

You should be in rowegrp.

You should also set up your account so that new files you create have the group set to rowegrp and the permissions should allow everyone in rowegrp to modify the files.

Subversion

We use Subversion to manage the source code. Probably the best way to use subversion (svn) is to use the svn+ssh method, which runs the svn protocol over an ssh tunnel.

The code is stored on the machine astrometry.net.

Here's how to test if it's working: 

   svn list svn+ssh://dstn@astrometry.net/svn

replacing dstn by your username at astrometry.net.

This should print: 

  branches/
  trunk/

It may ask for your astrometry.net password. You will probably want to set up RSA keys and your ssh config file so you don't get asked for your password all the time. (I know other people prefer to use ssh-agent or something like that; I usually just use unprotected RSA keys.) To do this (replacing dstn by your username, of course): 

  ssh dstn@astrometry.net
  cd ~/.ssh
  ssh-keygen -t rsa
  cat authorized_keys id_rsa.pub > tmp
  mv tmp authorized_keys

this will generate files id_rsa (your private key) and id_rsa.pub (your public key). By adding the public key to your authorized_keys file, you say that the associated private key can be used to log in.

Next, copy the private key to the ~/.ssh directory of the machine onto which you want to check out the code: 

  exit                (from astrometry.net)
  mkdir -p ~/.ssh     (it should already exist, I think)
  cd ~/.ssh
  scp dstn@astrometry.net:.ssh/id_rsa id_astrometry
  chmod 400 id_astrometry

Next, edit the file ~/.ssh/config and add the following: 

Host astrometry*
Hostname astrometry.net
IdentityFile ~/.ssh/id_astrometry
User dstn

Now, you should be able to type ssh astrometry to automatically ssh to astrometry.net without having to type your password.

Restricting automatic login

If you don't like the idea of being able to get to astrometry.net without having to type your password, you can restrict the use of your private key so that it can only be used to run subversion.

How it works is: in your authorized_keys file, in addition to the public key, you also restrict the key to be used for a particular command. For subversion, you restrict it to only the svnserve command.

On astrometry.net, edit your ~/.ssh/authorized_keys file. It will be one long line; make sure that whatever editor you use doesn't do a line-wrap or anything like that. The line will start with ssh-rsa and then a long string of digits: your public key. In front of the ssh-rsa string you need to add the restriction: 

command="svnserve -t -r /svn/"

so it will look like: 

command="svnserve -t -r /svn/" ssh-rsa AAA...

Getting the code

Okay, so now assuming that I've given you the right instructions and everything is working, you should be able to do this to check out most of the code: 

  mkdir astrometry     (or wherever you want to put the code)
  cd astrometry
  svn co svn+ssh://astrometry/svn/trunk/src/quads
  svn co svn+ssh://astrometry/svn/trunk/src/usnob-map

"co" means "checkout". If you haven't used subversion before, there's a pretty good manual here: http://svnbook.red-bean.com/ mainly you'll just need to do "stat", "up" and "commit"; we don't really use branching and merging yet because we don't have any released or production code.