PAN012: Build thread for Caltech

PAN012, the unit at Caltech was completed yesterday. It marks the end of the main build phase and took 8 weeks, which I believe is the fastest build completed to date! Id like to acknowledge Aru and Montu for their hard work during their internships. They did a fantastic job and should be very proud.

This particular build is even more significant because it is the first time the generation 3 electronics, based on using a single interface board to control the head unit and power distribution, has been implemented. Other improvements from the pre-2018 design include:

  • The head unit makes use of the new larger box, which will allow others to use different and larger cameras.
  • The mass of the head unit is balanced on the dovetail rail, which should improve observing stability (this remains to be seen).
  • The plates in the head unit have been simplified.
  • The pier is entirely new and extremely compact giving the counter weight more clearance.
  • Several other parts have been modified/simplified to reduce cost including the power supply,
  • The control box now has a removable platform making it a lot easier to fix things.
    This build allowed us to test a new refined parts list that James Synge kindly worked on and optimized. With this build we have now tested these new parts and refined the instructions for the entire build. In the next few weeks we will hopefully clean up the instructions and parts list in the hope of launching them in about 1 months time. This will greatly simplify the purchasing and building process for everyone from this point on.

Id like to thank Garreth Ruane for funding the build through NSF. Dimitri Mawet for funding Aru’s internship. Aru and Montu who worked tirelessly to get it done in an incredibly short time (especially as a lot of it was untested). James Synge for all his efforts on optimizing the parts list and instructions. Luc Boucher for the huge effort on the design on the new generation of control electronics and finally Wilfred for support with the control software. Without all these people this would not have been possible.

Please enjoy some of the cool images of the final robotic observatory. Hopefully there will be transit data in the not too distant future.


Well done!

Regarding the control box, I’d be a little concerned about placing the surge suppressor on top of the power supply: it is blocking the air vents used to cool the power supply.

While I wait for my phone to come back to life so I can call an uber home Ill write a quick summary of the night:

  • Tried to focus the cameras. First tried by eye. No chance. Even with the larger box the angle is really high. You can see the spot get brighter and dimmer as you focus, but the position where I convinced my self it was smallest and brightest was not when you inspect the image. So this technique is questionable.
  • Tried to focus using the images. We took images using the notebook, and manually adjusted the focus between each. Then opened 12 images and compared. This worked well but was super slow. We have images we can put on the drive to share with others to give us their opinion. Note, as the brightest object we could easily see, we chose to use Mars for this. Not sure if its too resolved for this???
  • With one camera focused, we decided to move to Polaris to try the Polar alignment. Nothing worked. See the next session which summarizes our night.
  • The comp could not get on the network when taken to the roof. It turned out that the static IP assigned to us wont work with the port on the roof. Once we switched to DHCP, it worked. We need to ask IT to fix this.
  • Despite the fact we were taking images fine in the lab, during the focus tests we had numerous issues. Firstly, the cameras were both responding and taking photos, but were not saving them. After trying the notebook instead of the terminal nothing changed. Finally, we rebooted the mount as we accidentally parked it, and when we did, we could now take images and save them. Not sure if there is a correlation but that was the only thing that changed.
  • When we could take and save images, despite the fact that the notebook was telling both to do so, the second one would work 10% of the time. No error would be spat out if it didnt it just simply chose not to do anything.
  • To do the manual focus test was laborious. This could be sped up with a script that simply takes an image, gives the user a prompt to refocus, then click continue when it would take another image, and then repeat 10 times to collect the images quicker. The slowest part was opening all 12 up, making the window smaller and then zooming in to the same region of the image to the same zoom so we could see the difference between the images. This again could be scripted.
  • Although polar alignment seem to run, it was not saving the first image taken in the home position. Also, although it was taking and image during the slew, and saving it, the image did not have arcs. It was like it was a static 1s image but named correctly as Rotation_east or whatever. We tried three times. Note, we did not have PAWS open. We inspected the saved images.

So all in all a very strange night of behavior from the cameras. These really need to run and run reliably if we are to attempt anything on-sky. @Aru and @Montu will focus their efforts them tomorrow. And, we need to get the polar_alignment doing what it should be. Help welcome.

1 Like

Quick summary of the night:

  1. We found the finder scope in the shaft of the RA access. I adjusted focus by looking at a random object and then adjusted the mount position to center Polaris on the reticle.
  2. We found out that one of the reasons why we couldnt see many stars was because the aperture was stopped down. We were at like f10-15 instead of f1.4. I thought aperture control was auto controlled not manual? Also, I noticed the aperture was nearly shut by looking through the front lens but confused this with the shutter. After playing with the lens I worked out how to set it manually and hey presto, we got a ton of light and could see stuff. THIS NEEDS TO GO IN THE INSTRUCTIONS We discovered this by taking manual 30s exposures and realizing we could see Polaris better with our naked eye. So we played with the camera settings until we discovered the issue.
  3. We discovered that unlike @wtgee response several posts above, you can NOT just put the cameras in manual mode and go for it. Once in manual mode you MUST rotate the little wheel to set the exposure time to BULB (on an SL1). It comes after 30". If you do this the computer will control the camera every-time. We deliberately moved it off this setting to another exposure time and the computer failed every time to connect to the camera. We put it back and both cameras worked fine. So the instructions must say, put camera in manual, and adjust exposure to bulb mode. Im sure there is some software that might be able to change this automatically but thats rolling the dice. Id set it in hardware to be sure.
  4. With the cameras actually taking images we could see, and under our control, we tried a polar alignment. We found that the sky is way to bright in LA. So seeing as though we can’t reduce the slew time (cause we want long arcs) or the exposure time, we used the aperture to reduce the amount of light getting in (note this affects both the arc brightness and the background). The algorithm failed every-time to detect anything, but we tried to manually walk Polaris towards the center of rotation of the RA axis. The issue is, Im not sure which star is Polairs to be honest. It was a long way from the center of the field if it was the one we were aligning to. We got close and the clouds rolled in. So didnt finish it, but took heaps of sequences so @Aru will share the images in the morning. We would appreciate input. Note, I think if we subtract the 30s exposure taken with the mount at home from the one when we are slewing, we should eliminate most of the background and be left with the difference, i.e. the arcs. This could work for bright sites. @Aru can test this tomorrow.
  5. We were about to start a short observation sequence but the clouds got us.
    Pretty successful night, but more work to do.
1 Like

My cameras are set to M, but I don’t recall setting them to BULB. I may simply have forgotten. Or something is wrong with our camera control. Yours are the first new cameras since mine went into operation last fall, so we could have accidentally broken something. I’ve recently purchased a used SL1 and used SL2, so can experiment locally with the camera control… though I’m focused on the documentation for the next month or so.

We had a very productive night last night. We had light overhead cloud which meant it was not ideal for photometric observations. However, with @wtgee help we worked through a bunch of items and took our first full observing sequence on a star that was actually having a transit (Wasp74). Here are the success’s we had:

  1. We set a single target - Wasp74b in the target list with a priority of 500. It accepted it slewed and tried to start observing. Except since one of the cameras did not work, as it has not been working from POCS_shell for the last two weeks, the observation timed out and parked itself. When we disabled auto detect of cameras and only provided the USB port for the one camera we wanted to use, it accepted it and began the observation and continued without issue. We took ~50 images, with 45 second exposure.
  2. The tracking was working really well. We were getting RA and Dec offsets of <10 arcsec (1 pixel) in both axes every time. In fact Id say the peak to peak was 8 ish arcsecs and it was more like 2-3 arcesec on average. So the polar alignment was done well. This also means plate solving is working well.
  3. With @wtgee help we got PAWS working and could see the latest images and other data which is great. We do not have all the sensors working at this point, but can fix this in future.
  4. We got VNC and ssh working fine, so we did the simulated observation remotely. This was a huge victory as it gives us confidence we can run things remotely now.
    Tonight we will try to observe HD 189733 b which is the brightest transit this week. Fingers crossed the clouds go away. Thanks for the help last night @wtgee!

Here is a single image of the field. Note the wispy clouds. Also, I tried uploading a fits and thats not an accepted file type so enjoy a single jpg instead.

1 Like

After collecting data through the entire transit of HD189733 on Tuesday night, Wilfred helped us do a quick data reduction. The results are shown here.

This shows the target PSF and the reconstructed PSF from various reference stars.

This is the light curve for the target with a moving average. The transit is clearly visible. This is the result of a preliminary reduction and we will continue to work on it over the next few weeks.

Very happy to say we built a PANOPTES unit from scratch in 8 weeks and got a transit detection in the next 2 weeks! This is a great outcome from Aru and Montus internship.


Just a follow-up with a slightly nicer plot. The red line in this plot is a model transit for HD189733, not an actual fit to the data (built with batman). Still has a nice residual to it.

Latest cut at the data. Correct batman curve and also curves from each color channel.

Great progress @nem.
What material did you use to protect the cameras in the head unit? We were thinking of using mosquito mesh but yours looks more solid and resistant to leaves. Thanks :+1:

For PAN008 I used some of the stiffer screen material used for windows. Should just be available from any hardware store, sold in rolls and you can buy a certain length of it. Not only was it stiffer but the hole size is a bit smaller.

I used the same material for the intake valves. I also cut out two pieces and put them at angles so the holes were even smaller.

I used aluminum meshing I found on Amazon. It was cheap and easy to cut with scissors. Here is the link to the product:

I plan to add this to the new parts list so others can just buy it.

I used a combination of mosquito net from a fabric store and aluminum window screen from a hardware store. I figured that the mosquito net would keep quite small bugs out, but not have a lot of stiffness, so combining the two seemed like it would provide the best of both worlds.

You can see a lot of pictures of my build of the bug protection in this album. For example:

Hi all!

Two cameras! TESS fields!

I connected to PAN012:

  • Started the vnc4server, which wasn’t running.
  • Updated all the POCS software (committed and saved all local edits under the aru_changes branch, changed to the develop branch, did a git fetch origin and then a git pull to put the code at develop@HEAD.
  • Killed some long-running jupyter process. Not sure what this was. Saw nothing open and running when I logged in. I’m using the panoptes user.
  • Took a picture with each camera (separately) using via $POCS/ <PORT> <EXPTIME> <FILENAME> to confirm that cameras were taking pictures. This worked fine but images are black. Lens cap on? @nem.
  • Updated the $POCS/conf_files/pocs_local.yaml file to include two entries, one for each camera. Set auto_detect: True. Changed primary: <NOTHING> to primary: 95cdbc.
  • Pointed the conf at a new targets file. In pocs_local.yaml changed the scheduler.fields_file to tess_sectors_south.yaml. Look ma! I can see some TESS fields from LA!
  • Started a jupyter-console in the terminal and used the (new) create_cameras_from_config to make the camera objects (see code below).
  • Created an Observatory object with simulators for everything except the cameras. Asked for an observation. Altered the time slightly.
  • Took pictures via the Observatory. I took two separate pictures, one at default of 120s, the other at 5s.
  • Profit! Except the images are black as per lens cap question above.

Here’s the code from the jupyter-console session. This is the same as a jupyter-notebook, just easier for me to do over ssh.

> jupyter-console

 In [1]: from import create_cameras_from_config

 In [2]: cameras = create_cameras_from_config()

 In [3]: from pocs.observatory import Observatory

 In [4]: obs = Observatory(simulator=['mount', 'weather', 'power', 'night'], cameras=cameras)
Out [4]: OrderedDict([('Cam00', < at 0x7f096e1f7b38>),
                      ('Cam01', < at 0x7f0976ec2048>)])

 In [5]:
Out [5]:  'Cam01'

 In [6]: obs.primary_camera.uid
Out [6]:  '95cdbc'

 In [7]: obs.get_observation()
Out [7]: <pocs.scheduler.observation.Observation at 0x7f0967e9b9b0>

 In [8]:
Out [8]:  'TESS_SEC04_CAM01'

 In [9]: obs.observe()
Out [9]:  {'Cam00': <threading.Event at 0x7f0965c92cf8>,
           'Cam01': <threading.Event at 0x7f0965c91c88>}

 In [10]: from astropy import units as u

 In [11]: obs.current_observation.exp_time = 5 * u.s

 In [12]: obs.observe()
Out [12]: {'Cam00': <threading.Event at 0x7f0965c8bf28>,
           'Cam01': <threading.Event at 0x7f0965cf6240>}

 In [13]: exit()

Assuming it’s just a lens cap issue, here are our beautiful TESS images. :slight_smile:

@nem @amukherj @Montu great job again! I could operate everything remotely just as I expected to. Instructions for starting up the VNC and all that were helpful.

Looks like any issue you were having might just have been some outdated code. We’ll obviously need to do a little more with the new arduino sketches but this sort of confirms it’s in a good state for that.

@nem the computer needs updating and a reboot but I didn’t want to do that as I’m not sure if it will connect to the same IP address and I didn’t want to lose access. Let me know when you are around and I’ll do it just in case I can’t reconnect.

I didn’t play with the mount or anything because I cannot see it. Is there a web cam hooked up?

Looking good!

Note also that I am always watching the log files when I do things with a unit. For a jupyter session that is:

> grc tail -F $PANDIR/logs/

where the grc gives you color output and the tail -F is “following” the log file.

Hi Wilfred
This is great! Thanks for logging in and doing that. Im glad to see that remote access was trivial. Aru indeed did a good job setting things up.

I was showing Luc the system yesterday around 2:30 pm PDT and noticed it seemed to have frozen. I just rebooted it, did the update as suggested and rebooted once more.

Yes lens caps were on, but I have removed them and the room lights are off now. I checked and the mount is away from anything it can hit (and I think the cable is sufficiently long enough). So now you can move the mount freely.

We will test that we get two images using the POCs shell hopefully later this week but you are welcome to log in in the mean time and complete your tests. Can you do me a favor, and please move any interesting commands and procedures that we dont currently have from the stuff you wrote above into the operation guide Aru wrote? It should take 5 minutes, but it would be nice to capture some of these commands for debugging. The ones that make most sense of course.

Hi everyone! I’m Therese, a summer intern who will be helping get PAN012 remotely deployed and ready to make some discoveries :raising_hand_woman:t2:

Just wanted to give some updates on the unit since it’s been a while:

A week ago we finished installing weatherproofing panels on our iEQ30 Pro mount. :umbrella: What we came up with was inspired by the PAN013 and PAN015 designs, and it looks like this after silicon-sealing:

That square window in the roof-like panel is temporary; it’s there so that the polar scope eyepiece can be used during polar alignment. Another square of aluminum will be siliconed on top to shut it afterward. We’ve digitized the dimensions of the final design, so a printable template will soon be added to the weatherproofing instruction set!

Last Friday, we also set PAN012 up on the roof of the astronomy building at Caltech. Yesterday we managed to get a decent polar alignment, and we’re going to observe from here over the next few nights to troubleshoot and make sure the unit operates smoothly on its own. The plan is to deploy PAN012 at its new home on Mt. Wilson next week; will keep this thread updated! :telescope::mountain:

1 Like

More updates!

We had PAN012 observe targets with known transits nearly every night last week. This gave us the chance to see whether the unit behaved as expected through the night even without us there to make adjustments. Specifically, we confirmed that the AC and weather interlock systems worked (if the power goes out or if clouds roll in while observing, the unit will park itself till conditions are safe again :electric_plug::cloud:). We also worked with @wtgee to fix and identify some bugs in POCS. Will post results of the transit observations once we get the chance to process the data! :alien:

After that, we packed the unit on Thursday and drove up to Mt. Wilson to deploy it at the site of the old Mark III interferometer behind CHARA. We got pretty far with setting PAN012 up on its new massive pier (!!) but we still need to make another trip there this week to get it connected to the internet.

Here’s some pics we took during the deployment!

Step 1: get a sick car :sunglasses::call_me_hand::fire:

digging trenches for the power and network connections :cowboy_hat_face:

I lifted the pier up there all by myself :tipping_hand_woman:

got the mount level :white_check_mark::eyes:


Introducing the hundred foot telescope on Mt. Wilson!!! :blush::telescope:


PAN012 says happy thanksgiving! :snowflake: :turkey: :snowflake:

1 Like

It’s been raining the past two weeks and unfortunately forecasted to continue the rest of the month, but the clouds cleared tonight so PAN012 did some observing :telescope:

I included a couple comets in its scheduler file, and here’s a view of one of them. It’s C/2019 Y4 alongside the galaxies M81 and M82, plus a meteor! This comet is faint now but projected to ramp up in brightness in the coming months, so we’ll keep our eyes on it :slightly_smiling_face::stars:

1 Like