Help getting started

#1

Good day

I attended a great short talk given by Sarah Caddy last Saturday at the Macquarie University Astronomy Open Night in Sydney and she presented an overview of the Huntsman telescope and her research on Galactic Merges.

She mentioned how there is a lot of work to be done on the software tracking of the telescope, and through browsing I found out the trackin software is based on the POCS software project.

I had a look at the GitHub repo and it looks very interesting but also a little daunting :slight_smile:

I’m a professional software developer based in Sydney working mainly on back-end systems for Financial Services. I also have an interest in Astronomy (albeit dormant since childhood) and I thought it could be an interesting project for me to get involved.

I’ve contributed to Open Source software projects before (including a small contribution to Discourse, which you are using to run this forum :)) but I don’t really know Python (I come from C/C++ but those days I mainly use Java, C#, Swift and various flavours of JavaScript) and my background in Astronomy is…not existent beyond Uni Physics (which I mostly forgot unfortunately).

I think I could easily handle installing the dev environment, and I’m also very familiar with Git. Learning Python should not be a problem. However, looking at the list of issues in GitHub I could not really find anything that looked simple or small enough for me to start having a look at to get myself familiar with the environment and software.

Would you have any suggestion on what I could start looking at that does not require much background knowledge?
Time permitting with my full time job I would love to potentially help out as I can.

Thanks heaps

#2

Hi Jeremy
I studied at Macquarie University and used to help run those Astronomy open nights when I was there 8 years ago. I moved to Hawaii and worked at Subaru where I helped on PANOPTES, in my spare time and met Wilfred who was an intern on the project and is now at Macquarie, doing a PhD on PANOPTES and specifically data reduction and extraction. Wilfred developed most of the control software for PANOPTES and is most familiar with it (along with James Synge). Its great to hear youre interested in the project. Hopefully, Wilfred/James will be able to navigate you through the software repo and set you onto an interesting task you could work on in your spare time. Wilfred is also sharing some of the software with Huntsman and last week he was up at Siding Spring testing some stuff on Huntsman.
I hope that gives you some context of the people in your area and who to reach out to.
Were all in the same boat in regards to your last point. Although some of us are professional astronomers, only Wilfred is on PANOPTES 100% of the time.

#3

Thanks for the great intro @nem! It’s appreciated and gives me some context.
I was really fascinated by the Huntsman project at the Open night!

#4

Jeremy,

Something that is not critical but will be really useful would be the ability to send a tweet for each of the “say” messages that the software posts; these are intended for human consumption, such as “Parking the mount” or “Observing HAT-9”.

I would suggest creating a python program in POCS/pocs/bin that uses the zeromq library (in use elsewhere in POCS) to listen for these messages. See the PanMessaging class, and the POCS.say method (it may be on a base class, I don’t recall).

It would be best to create an issue in POCS for this, and engage in a discussion with Wilfred and myself regarding the approach.

James

#5

Hi @james.synge

Thank you for the message.

I cloned the repo and followed the development instructions.

Unfortunately I only have an Ubuntu 18.04 VM and it has both Python 2.7 and Python 3.6.5 already installed so I had to be a little creative.

I’m not too familiar with Anaconda but I found using python3 and pip3 from the terminal seems to work well. I detailed the steps I had problems with below as it may help someone else.

The main problems I encountered (probably because of the above) were while executing ${POCS}/scripts/install/install-dependencies.sh.

I had to install MongoDB manually (due to different Ubuntu distro) and the script did not seem to install Conda or astrometry either.

While installing the dependencies I found these two differences:

libpng12-0 instead of libpng12-dev
and scikit-image instead of skimage

I then started the test suite and found out many tests failed because I did not have astrometry installed. I then followed the dependencies script steps manually and managed to have most tests succeed with only 2 failing tests. I can’t figure out why those 2 are failing but it does not look like because of missing dependencies so I assume the environment is ok now. I uploaded the test suite log output if you want to have a look (Discourse does not let me attach a text file):

pocs_test_log.txt

I’ll start having a look at the classes you suggested next week and see if I can figure it out how they work. Maybe as part of the feature could be useful to output to Slack too?

I can certainly create an issue for this.

Thanks

#6

Hi

I’m trying to get my bearings with the system and trying to watch those pocs.say messages.

I went to the bin directory and I can successfully start a session by doing:

>python3 pocs_shell
Welcome to POCS Shell! Type ? for help
POCS > setup_pocs simulate all
POCS > run_pocs
Starting messaging
Command publisher started on port 6500
Command subscriber started on port 6501
Message subscriber started on port 6511
Message publisher started on port 6510
Starting POCS - Press Ctrl-c to interrupt

However going with the browser to http://localhost:6511/ does not bring any messaging page, I get ERR_INVALID_HTTP_RESPONSE.

I can see some log messages in the log directory and I successfully executed
python3 -m pytest pocs/tests/test_messaging.py -v

Am I on the right track?

Thanks

#7

Sounds like you are off to a good start.

The messaging ports are used for zeromq (e.g. pub-sub), not for HTTP & HTML.

A messaging client would listen to port 6511 in order to receive all the POCS.say messages.

#8

Great thanks @james.synge that makes sense.
I’ll see if I can write a client to listen on that port then as a start

Cheers

#9

Hey @james.synge

Progressing well :slight_smile:

I created a simple Python script that uses the PanMessaging class to subscribe to the PANCHAT channel.

I used the Tweepy library (http://www.tweepy.org/) to output the messages on a Twitter account I have for development, you can see them there:

https://twitter.com/tweet_ispgoz

I think it’s good enough as POC, I will create the GitHub issue now and see which requirements you have (i.e. config files, logging levels, channels etc.)

Also, my POCS installation keeps core dumping after a while (it’s fully simulated).
I always get this error after a few minutes:

Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x7f10380cf940>>
Traceback (most recent call last):
  File "/usr/lib/python3.6/tkinter/__init__.py", line 3504, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x7f103845dbe0>>
Traceback (most recent call last):
  File "/usr/lib/python3.6/tkinter/__init__.py", line 3504, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x7f10384c0160>>
Traceback (most recent call last):
  File "/usr/lib/python3.6/tkinter/__init__.py", line 3504, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Tcl_AsyncDelete: cannot find async handler
Aborted (core dumped)

Any idea?
Thanks

#10

HI @jeremylan, sorry for not getting back to you earlier.

That is an odd error but I have a feeling in the back of my brain that it is related to the default backend for matplotlib, which is used for creating some plots at various points in the operation. By default matplotlib will try to bring up an interactive graphical environment when it is called, which is not something you want to do while running it in a script. This suspicion mostly comes from the fact that those errors involve Tk stuff, which is what is used for those interactive GUIs, and which we certainly do not use anywhere within the code.

The link above has steps for customizing your environment, namely via the matplotlibrc file. You want to set backend: Agg, which will render them as png or jpg files appropriately. Agg is builtin with matplotlib so nothing else should need to be installed.

@james.synge, we had various conversations about this backend and the install script. Is it true that we currently expect the user to set this manually? If so we clearly need to get that step more visible.

#11

Actually, we do set this. There is a matplotlibrc file in $POCS.

@jeremylan are you invoking the script from within the $POCS dir itself? i.e. cd $POCS then bin/pocs_shell. That might not be obvious so I could see you invoking it from somewhere else in which case the $POCS/matplotlibrc would be ignored. We expect all commands and scripts to always be run directly from $POCS.

#12

Blockquote
@jeremylan are you invoking the script from within the $POCS dir itself? i.e. cd $POCS then bin/pocs_shell. That might not be obvious so I could see you invoking it from somewhere else in which case the $POCS/matplotlibrc would be ignored. We expect all commands and scripts to always be run directly from $POCS.

Hi @wtgee yes, I was running the script from the bin directory, sorry :flushed:
I think that was the problem, it’s now running OK and I get just a warning, it must have been the one triggering the core dump:

/home/jeremylan/.local/lib/python3.6/site-packages/astropy/visualization/stretch.py:264: RuntimeWarning: invalid value encountered in log
  np.log(values, out=values)
/home/jeremylan/.local/lib/python3.6/site-packages/matplotlib/colors.py:504: RuntimeWarning: invalid value encountered in less
  xa[xa < 0] = -1

All good then thank you very much

#13

Glad to hear that fixed it and it’s a good reminder that we need to make the docs more explicit.

Cheers!

#14

PR created

#15

Thanks to Jeremy, we now have Twitter support in POCS. I’ve asked him to help with instructions for setting this up, which I can then test on PAN006.