PAN010: Build thread for UofA

[nem]

Nice plot.

Ok is the NUC tripping while connected to anything? Or even when it is fully isolated? What power supply are you using? The NUC supplied power supply? Does it trip with the NUC provided power supply?

If the answer is that it trips when not connected to anything using the NUC provided power supply, then you have an issue with the NUC and Id send it for RMA to Intel. If however, you are using anything after market when it trips (you have your own power supply for example) and/or you have other things connected to it that could be causing it to trip, then the fault may still be with your setup. Id also google what causes thermal trips in NUCs and see if there is something obvious.

Independent of the NUC, can you communicate with either arduino through ANY computer at UofA? If not, Id google this condition and see what the errors could be. But it may be fastest to buy two new arduinos, and plug them directly into a computer before soldering anything and confirm they work. Then replace the ones you have if they do.

Hope this helps.
N

[jmk1729]

We solved the NUC processor thermal tripping. It works as intended with its 19 V power supply, however when the power board is used as a 12 V source for the NUC, it trips almost every time after loading to the login screen. It is simply not enough power, so following @james.synge lead, we adjusted our PSU voltage pot to output 13.6 V. This is within the lower range of acceptable voltages for the NUC, and it allows the backup battery to charge at a rate above the minimum voltage required per cell.

Now each of the power board lines outputs 13.6 V and the NUC performance is hardware limited. We can run about 12 YouTube videos simultaneously to reach 100% capacity on all four CPU cores. We will stress test it more when we’ve finished with tuning the AC sensor and are preparing for rooftop deployment in the next few weeks.

Communication between the Arduino micros and the NUC is still unavailable. @wtgee suggested we upload the Arduino scripts using a separate computer, then use them with the NUC. This functions as a workaround, but we will continue to investigate.

Here is a quick rundown of the “diagnostic” tests we’ve performed: we have successfully uploaded and ran scripts onto a bare Arduino micro using the NUC. We have also successfully uploaded and ran scripts for the camera and telemetry boards using three separate computers: one desktop and two laptops. When we try to use the NUC to upload a script to the camera or telemetry boards, it’s always avrdude that reports an inability to interface with the micro. One of the next steps could include pulling out the Arduino micros from the boards and testing if the NUC can interface with them.

Regards,

Justin

[james.synge]

I found that uploading a new sketch from the NUC to the Arduino was unreliable, but a Windows laptop was more reliable (this was in 2017). I’ve recently been doing some Arduino development, and encountered something odd: I could upload the Blink sketch (very small) to the Arduino from my Linux laptop, but not a complex program (nearly 32KB). It would almost always fail.

I searched for hints, and tried many things to fix it. One that seemed ridiculous was to change the USB cable, but eventually I gave that a try and was surprised to find that it worked, and worked well. The suggestion I read is that a lower quality cable will have thinner wires, so it can’t carry as much current without the voltage dropping noticeably; storing the new sketch into the flash memory takes more power, i.e. more current, which drops the voltage. And writing a longer sketch to flash makes this worse (i.e. the voltage will drop lower). So, a crappy USB cable will work fine with simple (short) Arduino sketches, but may fail as the programs get longer. Sigh.

Note that I haven’t validated that explanation by measuring the voltage during flashing.

[wtgee]

Sort of not surprising about the cables. Stupid USB.

Curious if everyone is having the same experience with the arduino command line program to upload? It wouldn’t change things like the cabling issue obviously but seems to be a little less finicky for me.

I’d prefer to go the command line option as then we can just automate the upload and don’t have to bother with users installing the IDE.

The other option is to use the online IDE (although I think @james.synge said that didn’t work with a chromebook). Using the online option someone can essentially just copy over from the projectpanoptes accounts (via a “share” kind of feature) and then upload, so they wouldn’t have to deal with where the files are placed and the stupidity regarding folder structure that the normal IDE seems to impose.

[jmk1729]

Hi all,

1. We will try @james.synge and @wtgee’s suggestions regarding sketch uploads from the NUC to an Arduino.

2. We tuned the AC sensor on the power board. @lucboucher explained the need for a realistic current draw during the calibration process, so we used our PSU and a 10 Ohm power resistor to pull about 1 A to set the threshold voltage on the comparator. This works, albeit we need to test that when we turn off mains power, POCS reports the problem and shuts down the unit.

3. We are on the roof of Steward Observatory this week. I believe we succeeded in polar aligning our mount last night in spite of some cloud coverage. The process was pretty straightforward since @nem told us about the finder scope - it also helps that we’re not in LA. Here are some setup pictures:

SETTING UP THE UNIT

GOOD THING WE DON’T NEED TO LOOK SOUTH

And the polar alignment iteration we stopped on:

CAM 0 (the one used to solve for the celestial poles)

CAM 1

ROTATION OF CAM 0

ROTATION OF CAM 1

CENTER OVERLAY

Assuming this polar alignment is satisfactory and, if necessary, we just have to tweak camera focus or other lens parameters, the next step is to set up an observatory in POCS and schedule an observation. We want to track a known transit much like PAN012 did.

We don’t have internet access on the rooftop as we initially predicted. This may change, but for now we’re just going to have to save all of our data onto an external hard drive and upload them to the Google cloud storage later.

[nem]

Nice one. We aligned it till the track disappeared or was super small. So keep tweaking the mount manually.

If you don’t have network, you will have to manually start the data acquisition and go back and check it didn’t crash. Make sure to get up there early to make sure its in a safe state cause if its left looking east ward, cause of a crash, and the sun rises the camera could get damaged.

[wtgee]

Awesome work! Looks like you didn’t have any problems with the polar alignment script. Did you try to do any tracking of stars or anything after that?

[jmk1729]

No, but we are attempting tracking and scheduling tonight. We are using an old version of POCS, so we are considering doing an update before we get back out on the rooftop.

[Jonah.Rolfness]

I have some information about the arduino uploading and my experience with it. I was going to put it in my update post, but haven’t had the time to finish that yet. Sorry! Maybe I could’ve saved some trouble.

I wasn’t able to get the arduino IDE working using the terminal install with apt-get. The option to select a serial port was greyed out, and no troubleshooting fixed it. After digging a bit online someone said you had to use the website IDE, so I reinstalled and had success with that. As far as I remember the only other thing I had to do was add myself to the dialout group using sudo usermod -a -G dialout $USER. So far I’ve been able to upload the camera_board.ino sketch just fine each time I’ve tried and was able to read data.

I’ll see if I can find the post that tipped me off the using the website IDE again for you all. Maybe it has info regarding why the terminal install fails.

[wtgee]

The permission errors are also described here: Permission problems while connecting to arduino

[atrodack]

Hi all,

We have been continuing our work up on the roof of Steward Observatory with trying to track stars and schedule a target for observing a transit. We have run in to some things that we were hoping we could get some advice on. I will attach some pictures below to illustrate.

Our Polar Alignment to start our night of observing:

1. After some experimenting, we figured out we can change the exposure time and number of exposures run_pocs uses by adding keywords into the targets yaml file in $POCS/resources/targets/. Is there a good rule of thumb for how to set these numbers for a given target beyond making sure you cover the full transit (ie exposure time * min number of exposures >= total time of transit), and don’t over-expose the images?

2. Is there a way to set the start time of the observation in the configuration so that run_pocs can be left running waiting for the transit to start, and automatically start observing rather than us having to start run_pocs manually at the correct time?

3. When we pick our own target, we follow the formatting in simple.yaml to create our target list that has 1 target star that we found has a transit. We then point pocs_local.yaml to look at that file, and run the pocs shell. From there we “run setup_pocs night weather” and then “run_pocs”. Once it starts observing our target, it doesn’t appear to track the star while exposing the cameras (see 120s exposure below that we did to make clear if we were tracking or not, so it is a little overexposed). How do we make sure that the mount tracks the star? Is there something we need to include in our target or pocs_local config? We tried having the pointing auto-correct both True and False and that didn’t seem to matter for this.
   

   

4. It is our goal to track a transit and get the data collected so we can ask for help reducing it to verify we see an accurate light curve. Obviously we need to solve the previous questions to get to that point, but we would like to try doing the data collection by the end of this upcoming weekend. @wtgee can you help us pick a quality target and understand how to track it (mostly just questions 1-3 above), and then maybe help us do the data reduction following if our data collection goes well?

[atrodack]

Here is a picture when it did appear to be tracking at the correct speed

[atrodack]

We are using the version of POCS with this commit key: 4a3d30545fd4608d58b806c8f256a09f2933b169

[jmk1729]

@james.synge, @wtgee, I can’t upload the log files directly, so I’ve hyperlinked to them from a google drive folder. The links are labeled the same as their corresponding images Alex posted above (post 34 and post 35).

20190408T043207.jpg - this is a transit we were actually trying to track. We picked it using the transit calculator.

20190407T073437.jpg - this is a transit selected from simple.yaml.

Clearly the tracking is off when trying to set our own observation target. Both observations have a 120 second exposure time, but the image from 04/07/2019 uses f/5.6 while the one from 04/08/2019 uses f/2.8.

[jlumbres]

The UA team has been pretty busy these past several weeks, especially since Olivier was visiting UA for 5 days (a new record in over a year!). Most of the content has been mentioned in the weekly meetings and on slack, but I’ll reiterate it for the log.

Observation Attempt (Sunday 2019/04/21):

• Olivier was there to join us for most of our time on the roof! We did a polar alignment and took some pictures of M51.
  – I’m told “pictures or it didn’t happen” and given Olivier’s wavefunction equation, I present the obligatory embarrassing adviser photo: Olivier using the trackpad on my laptop wrangling ds9
  

  

• There wasn’t a good transit for the night, so we decided that we’ll do an overnight test to check that the PANOPTES unit will shut itself off correctly. Past tests have shown to us that POCS checks the time and horizon status to determine shut down. Only the weather was simulated in POCS.

• The next morning when I came in to check on the unit, something went astray and it could not park itself correctly. The good news is that it didn’t point east towards the sunrise.
  

  
  
  I manually moved the unit back to park, collected the data and logs, then shut down the unit.
  
  

• We shared the logs with @wtgee. It seems like there wasn’t any error, just the serial device stopped communicating. It is possible that we bumped a cable while maneuvering the night before in the darkness and packing it up for overnight observations. So that is something we should be careful about in future testing.

Observation Attempt (Monday 2019/04/22):

• We discussed the night before to keep the entire PANOPTES unit on the roof for the duration of the week. The forecast predicted cloudy skies for Monday night, so we did not cover the unit. Given last night’s observation results, we decided to keep the unit off until the next observation attempt.
• However, the forecast proved to be slightly untrue and there was a bit of rain. I got ahold of last minute roof access to cover the unit with tarp:
  
  

Observation Attempt (Tuesday 2019/04/23)

• Very cloudy day, but forecast predicted the clouds would clear out late in the evening. The original plan was to inspect the unit, polar align it again, and leave it alone to track a transit. Olivier suggested that we’ll call the decision at 10 PM since the transit was to begin at 12:30 AM. When Justin and I went to the roof, the seeing was very bad (the stars twinkled so strongly, Sirius was pretty much a strobe light) and not suitable for photometry. Test run cancelled for this night.

Wednesday’s observation attempt (2019/04/24):

• We had weird mounting issues, where it behaved like it was unbalanced? (It’s always something new on the roof.)
  – The mount made a loud groaning gear noise, probably a load noise.
  – When we went from unpark, park, unpark, go_home, sometimes only one axis on the mount would move correctly and the other not so much, if at all.
  – The mount worked well with the controller, but it was not with POCS. We contacted Wilfred about it and dragged Olivier up to the roof from the lab.
  – We can probably contact iOptron about this? This is the second mount issue we’ve come across.

• After the mount started to act like it was working, we started tracking a transit even if late. However, the transit we picked that night required a meridian flip, so POCS vetoed it. We overran it by adjusting some parameters in the yaml file.

• We analyzed our previous 120 second exposure data of M51 to study the star magnitude limits we could handle observing from the Steward Observatory roof. We used ds9 to check the star intensity counts and Stellarium to determine the star mapping and magnitude. A magnitude 11 star was about 100 counts above the noise floor. Therefore, any stars for observing transits from the Steward Observatory roof will have to be no fainter than magnitude 10.

• We left the unit operating overnight. Justin came in the morning to check on it, it parked itself correctly.
  

  

• Justin also pulled the data from the harddrive. Unfortunately, we couldn’t do any data reduction because of the meridian flip.

Friday (2019/04/26):

• We pulled the unit down from the roof. We are going to do some indoor work on it (see below). The unit is currently living in Justin/Olivier’s shared office. We are aiming to be ready for observing HD 189733b on May 23 at the roof of Steward. We will be leaving the unit on the roof for a bit and have it run autonomously for several days (weather pending) to ensure confidence at leaving it up on Mt Lemmon.

Wednesday (2019/05/01):

• Olivier and I had a telecon with Alan Strauss (Mt Lemmon Sky Center director) and Don McCarthy (SO faculty) for discussing how to deploy our unit onto Mt Lemmon.
  – We ultimately chose Mt Lemmon over Kitt Peak because Mt Lemmon Sky Center is maintained by Steward Observatory, which will make the internet/power costs, accessibility, and approval much easier. Kitt Peak is a multi-university area similar to Mt Wilson, so despite darker skies, it will be more difficult at this stage of PAN010.
• The mountain ops head couldn’t make it to the meeting, but Alan asked some basic questions about PANOPTES.
• Alan has started the email chain involving the mountain operations group and other key people (e.g. Buell Januzzi, Director of Steward Observatory). Not sure how long it’ll take for this to get approved though.

What’s up next?

• Figure out what’s wrong with the mount and if we need to contact iOptron
• Weather proofing
  – Protecting the mount
  – Acquiring USB and electrical cables to handle outdoors
  – Soldering down cable wire to the connectors (they’re currently only screwed in)
• Updating POCS codes
• Getting weather sensor to work
• Image coadding to make pretty pictures

[wtgee]

Thanks @jlumbres for the write-up, very helpful!.

Regarding the mount gear noise, was this during a polar alignment test or during normal operations? If you have time in the lab and the unit is set up it would be nice to see if it is still making that noise.

@james.synge is also setting up PAN005 for cloud access so I’ve asked him to check on PAN010 permissions at the same time. We can test the cloud access at any time and should do it before putting it up for next observation.

What’s the status of the weather station? Is that just something that needs to be done or were there problems with it?

[jmk1729]

The mount struggled, or outright failed, at moving the RA axis in two situations. The first is moving from park to home, then back to park. In this case it failed consistently returning to park from home for a block of time.

The second situation is during polar alignment, typically when capturing an image while rotating from east to west. This is when the gear noise happened as I recall.

We can play around with the mount when we find some time. Same goes for setting up for the google cloud storage.

The weather station needs to be integrated into the system, meaning we need to understand what it’s doing and how its sensors are being read from PEAS/POCS.

[wtgee]

Does sound like a balance issue so that’s the first thing I would check. Make sure all the wires are plugged in when doing that as it can add just enough weight.

[jlumbres]

UA PAN010 Update from our work today:

• We connected to the Google Cloud server.

• We’re in the process of getting ssh access for roof testing. It’ll be good for us when we’re ready to go to park it on the roof.

• We uploaded the telemetry and camera boards arduino scripts directly using @atrodack’s laptop and the most recent arduino gui version.
  – We tried in the past using the cloud version, but it didn’t work.
  – On the camera board, the accelerometer and humidity sensors worked in both the arduino gui and PEAS. However, the humidity sensor was finicky and moving something caused NaN values to output. We’re not sure what is going on there, so any suggestions welcomed.
  – For the telemetry board, the humidity sensor worked in both the arduino gui and PEAS. However, we’re not sure what’s going on with the temperature sensors (they output -127.0 values). I suspect it could be a funny way I wired the temperature sensors on the telemetry board. I’ll go investigate it.

• We managed to get the cloud sensor to be detected in PEAS (took a while, though…)
  – We found that having the arduino scripts uploaded gets PEAS to see the cloud sensor.
  – We hung out the cloud sensor outside the window in Justin/Olivier’s office. The weather stayed at “Very Cloudy” (it is indeed very cloudy today here in Tucson) and the sky temperature increased. We suspect that some of our results are skewed because it’s detecting parts of the building. We’ll need to test this on the roof.

• We set the mount’s meridian position limit to 25 degrees using the controller.

• We’re in the process for setting up the weatherproofing.
  – We have access to a 3D resin printer (through CAAO/LBTI) that we’re going to use for the camera box seals (featured from WISRD). We’re going to go through some test versions on another 3D printer to make sure we get the fit correctly before going through the resin printing. We can’t use a lot of the resin, so we decided to use it for the camera box seals.
  – We’re making a list of equipment we need to procure.

[nem]

Great to see the UofA unit in action.

In regards to setting the exposure time, we did this for HD189733 by taking an exposure, and checking the counts (in ds9 I believe). Aru showed me that you can type the name of a target into Ds9 and it will find it in the field so you dont need to look for it manually. From last Augusts efforts we worked out that the camera has a 2048 count pedestal (you cant get less counts than this) and a maximum of 13???. So you want the signal from the target to be between these two values. About 9000-10000 counts would be perfect but it will vary as the star moves across the pixels and has a different count in each color channel. So we adjusted things until we go 8000-10000 counts in the brightest channel and took a few images to see how much it fluctuated as it walked across pixels. I think in the end we went with a 35 second exposure time with the SL1 for this target and an open aperture. So this should work for you too.

Good luck