PAN014: Build Thread/Questions - Sierra College Astronomy Club

Glad it is working a bit better. Sorry the next steps weren’t as obvious, I will go do some instruction updating now and let you know in an hour or when done.

What did the error about the cameras look like and what step produced that error?

I’ve updated the above document. Note that this is for true automated operation, which might not necessarily be what you want for testing. I’ll be working on an additional document (or jupyter notebook) today that would go through some more manual control.

Let me know about any comments. Thanks!

I had issues with the camera when I was in the POCS shell

POCS > setup_pocs
Problem setting up POCS: CameraNotFound: No cameras available

However with your more complete setup, I started with the PAWS shell and

PEAS > load_weather
Loading AAG Cloud Sensor on /dev/ttyUSB1
PEAS > load_control_board
Loading control board sensor
PEAS > load_camera_board
Loading camera board sensor
PEAS > start
Starting sensors
Problem storing captured data: AttributeError("‘AAGCloudSensor’ object has no attribute ‘name’")

But then when I checked the last_readings

PEAS > last_reading weather

WEATHER:
{’_id’: ‘ce8b9519-8823-4d2f-a1b9-23624d6abda0’,
‘data’: {‘safe’: True},
‘date’: datetime.datetime(2016, 9, 9, 8, 0, 13),
‘type’: ‘weather’}

1530552.2 minutes old
PEAS > last_reading camera_board
/var/panoptes/panoptes-utils/panoptes/utils/database/init.py:60: UserWarning: No record found for camera_board
warn(*args)
No reading found for ‘camera_board’
PEAS > last_reading control_board
/var/panoptes/panoptes-utils/panoptes/utils/database/init.py:60: UserWarning: No record found for control_board
warn(*args)
No reading found for ‘control_board’

So I don’t think the cameras or control boards are being read in…

1 Like

I have attached the weather sensor now and the errors I get with not finding the weather stations are gone. However, when I’m in the PEAS shell and run load_weather it says it find the weather station, but then kicks me out often PEAS shell back into the Linux. Loading the camera and control board seems to load in fine.

Still even after all this setup_pocs in the POCS does not find the camera, though it does not detect it:

I0810 00:29:46.370 logger.py:281] ****************************** Starting PanLogger ******************************
D0810 00:30:05.939 init.py:084] Camera config: {‘auto_detect’: True, ‘devices’: [{‘model’: ‘canon_gphoto2’}, {‘model’: ‘canon_gphoto2’}], ‘primary’: ‘14d3bd’}
D0810 00:30:05.939 init.py:092] Auto-detecting ports for cameras
D0810 00:30:07.714 init.py:102] Detected Ports: [‘usb:002,013’, ‘usb:002,012’]
D0810 00:30:07.715 init.py:136] Creating camera: canon_gphoto2
D0810 00:30:07.716 init.py:140] Camera module: <module ‘pocs.camera.canon_gphoto2’ from ‘/var/panoptes/POCS/pocs/camera/canon_gphoto2.py’>
D0810 00:30:07.766 camera.py:096] Camera created: Cam00 (XXXXXX) on usb:002,012
D0810 00:30:07.766 camera.py:724] GPhoto2 camera Cam00 created on usb:002,012
D0810 00:30:07.766 canon_gphoto2.py:025] Connecting GPhoto2 camera
D0810 00:30:07.766 canon_gphoto2.py:034] Connecting to camera
D0810 00:30:07.766 camera.py:745] gphoto2 command: [’/usr/local/bin/gphoto2’, ‘–port’, ‘usb:002,012’, ‘–get-config’, ‘serialnumber’]
D0810 00:30:07.771 camera.py:768] Getting output from proc 56
E0810 00:30:10.294 error.py:013] CameraNotFound: Camera not responding: Cam00 (XXXXXX) on usb:002,012
E0810 00:30:10.295 init.py:144] Cannot find camera module with config: {‘model’: ‘canon_gphoto2’, ‘name’: ‘Cam00’, ‘port’: ‘usb:002,012’}
D0810 00:30:10.295 init.py:136] Creating camera: canon_gphoto2
D0810 00:30:10.296 init.py:140] Camera module: <module ‘pocs.camera.canon_gphoto2’ from ‘/var/panoptes/POCS/pocs/camera/canon_gphoto2.py’>
D0810 00:30:10.354 camera.py:096] Camera created: Cam01 (XXXXXX) on usb:002,013
D0810 00:30:10.354 camera.py:724] GPhoto2 camera Cam01 created on usb:002,013
D0810 00:30:10.354 canon_gphoto2.py:025] Connecting GPhoto2 camera
D0810 00:30:10.354 canon_gphoto2.py:034] Connecting to camera
D0810 00:30:10.354 camera.py:745] gphoto2 command: [’/usr/local/bin/gphoto2’, ‘–port’, ‘usb:002,013’, ‘–get-config’, ‘serialnumber’]
D0810 00:30:10.359 camera.py:768] Getting output from proc 57
E0810 00:30:12.890 error.py:013] CameraNotFound: Camera not responding: Cam01 (XXXXXX) on usb:002,013
E0810 00:30:12.890 init.py:144] Cannot find camera module with config: {‘model’: ‘canon_gphoto2’, ‘name’: ‘Cam01’, ‘port’: ‘usb:002,013’}
E0810 00:30:12.891 error.py:013] CameraNotFound: No cameras available

We’ve found a problem with the camera board which required us to add an extra voltage regulator.
Now we seem to be having issues with gphoto2 which when running in the background seems to prevent us from talking to the cameras. We can kill gphoto2 and then it communicates with camera properly when we run pocs_setup
Now the issue is when we run pocs_run it claims that there is no AC power to the cameras which is certainly not the case. No sure what to do next …

Ok. We got around this problem by coding out the AC power check. Now things seem to be working. POCS seems to be running a slewing and picture taking routine after I set weather and night to simulation in setup_pocs

Also load_weather does not work. It kicks me out of the PEAS shell and never load. any idea what is going on?

Now we have finally got everything working - as far as I can tell. The weather station seems to be good. We just needed to get power to it!

One thing that seems to be an issue is that gphoto2 seems to be already running when I first run setup_pocs. We have to kill -9 the programs to get them to find the cameras, otherwise the cameras are not found…

Hello! We were having some difficulties with the software, so we reinstalled the Dockers and POCS and tried to run the software tests and got the following which stopped the testing from happening at about 44% through the process. Any idea what it going on?

`pocs/tests/test_constraints.py::test_default_weight PASSED [ 43%]
pocs/tests/test_constraints.py::test_altitude_subclass PASSED [ 44%]
pocs/tests/test_constraints.py::test_altitude_no_minimum PASSED [ 44%]
pocs/tests/test_constraints.py::test_altitude_bad_param PASSED [ 44%]
pocs/tests/test_constraints.py::test_basic_altitude ERROR [ 44%]

===================================================== ERRORS ======================================================
______________________________________ ERROR at setup of test_basic_altitude ______________________________________

dynamic_config_server = None, config_port = ‘4861’

@pytest.fixture(scope='function')
def horizon_line(dynamic_config_server, config_port):
  obstruction_list = get_config('location.obstructions', default=list(), port=config_port)

pocs/tests/test_constraints.py:34:


key = ‘location.obstructions’, host = ‘localhost’, port = ‘4861’, parse = True, default = []

def get_config(key=None, host='localhost', port='6563', parse=True, default=None):
    """Get a config item from the config server.

    Return the config entry for the given `key`. If `key=None` (default), return
    the entire config.

    Nested keys can be specified as a string, as per [scalpl](https://pypi.org/project/scalpl/).

    Examples:
        >>> get_config(key='name')
        'Generic PANOPTES Unit'
        >>> get_config(key='location.horizon')
        <Quantity 30. deg>
        >>> get_config(key='location.horizon', parse=False)
        '30.0 deg'
        >>> get_config(key='cameras.devices[1].model')
        'canon_gphoto2'
        >>> # Returns `None` if key is not found
        >>> foobar = get_config('foobar')
        >>> foobar is None
        True
        >>> get_config('foobar', default='baz')
        'baz'
        >>> # Can use Quantities as well
        >>> from astropy import units as u
        >>> get_config('foobar', default=42 * u.meter)
        <Quantity 42. m>

    Args:
        key (str): The key to update, see Examples in `get_config` for details.
        host (str, optional): The config server host, defaults to '127.0.0.1'.
        port (str, optional): The config server port, defaults to 6563.
        parse (bool, optional): If response should be parsed by
            '~panoptes.utils.serializers.from_json', default True.
        default (str, optional): The config server port, defaults to 6563.

    Returns:
        dict: The corresponding config entry.

    Raises:
        Exception: Raised if the config server is not available.
    """
    url = f'http://{host}:{port}/get-config'

    config_entry = default

    try:
        response = requests.post(url, json={'key': key})
    except Exception as e:            get_root_logger().info(f'Problem with get_config: {e!r}')
    else:
        if not response.ok:
          raise Exception(f'Cannot access config server: {response.content}')

E Exception: Cannot access config server: b’\n500 Internal Server Error\n

Internal Server Error

\n

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

\n’

/opt/conda/lib/python3.7/site-packages/panoptes/utils/config/client.py:58: Exception
---------------------------------------------- Captured stdout setup ----------------------------------------------

  • Serving Flask app “panoptes.utils.config.server” (lazy loading)
  • Environment: production
    WARNING: This is a development server. Do not use it in a production deployment.
    Use a production WSGI server instead.
  • Debug mode: off
    ----------------------------------------------- Captured log setup ------------------------------------------------
    CRITICAL panoptes:conftest.py:275 Starting config_server for testing function
    INFO panoptes:conftest.py:288 config_server started with PID=4186
    INFO panoptes:conftest.py:296 Setting testing name and unit_id to PAN000
    INFO panoptes:conftest.py:300 Setting testing database to panoptes_testing
    INFO panoptes:conftest.py:304 Setting testing scheduler fields_file to simulator.yaml
    INFO panoptes:conftest.py:308 Setting temporary image directory for testing
    INFO panoptes:conftest.py:313 Setting all hardware to use simulators: [‘camera’, ‘dome’, ‘mount’, ‘night’, ‘er’, ‘sensors’, ‘theskyx’, ‘weather’]
    `

Hi @sierra_astronomyclub,
This looks similar to the config issue we were having over at PAN015 with our AAG cloud sensor. In pocs_local.yaml @wtgee had us add an endpoint url. It doesn’t look like this test is using this specific information in the test, but the server could be trying to start without an endpoint url or trying to call the config function to get it.

Additionally, if you are planning to use the AAG cloud sensor you’ll need to update pocs_local.yaml anyway. The url is put in the environment sensors and above the weather station section. The environment sensor section should be:

environment:
  auto_detect: false
  camera_board:
    serial_port: /dev/ttyACM1
  control_board:
    serial_port: /dev/ttyACM0
  weather:
    url: http://127.0.0.1:5000/latest.json

Adjust the Arduino ports as necessary. Hope that helps.

Thanks for your help. We are still having issues with the part of the process. We have summarized our errors and what we can resolve in these documents: https://docs.google.com/document/d/1AXzAL8Z5uDeeFF8WW83Y_xNvuIlXDje8OB9JBRBW4MA/edit?usp=sharing

I’m not sure if the test files are supported with the docker version of POCS at this time. Do you have control of the peas-shell and pocs-shell?

I’m wondering because the instructions we used to download the docker version did not tell us to use the test scripts. We used this document, written by @tmcook. What download process are you using?

Ok. After starting from scratch and reinstalling Ubuntu and then the PANOPTES software, we’ve managed to get the mount moving with the run_pocs command. I hope we are ready to start testing this telescope out in the field.

Some of the issues we had was the stopping of gphoto2 before running setup_pocs, otherwise the cameras would not be found. We also had to bypass the power check in core.py , otherwise the program would always claim the power was not on.

The next challenge may be the weather sensor and it’s operation. We are not really seeing a response to any outside influence such as wind …

Thanks, we actually decided to reinstall Ubuntu and the PANOPTES software and see if we will have better luck.

I’m not sure what you mean by the “stopping” of gphoto2, can you explain more?

Which version of the electronics board are you using?

Are any of the values changing, like the temperature?

" I’m not sure what you mean by the “stopping” of gphoto2, can you explain more?"

For some reason (we haven’t figured out why yet), each time the device reboots there’s 2 instances of gvfsd-gphoto2 running, and taking control of the camera devices on the system. Until they are killed, trying to start gphoto for the cameras gives an error about being unable to claim the usb devices. See the first error on the second page here: https://docs.google.com/document/d/1AXzAL8Z5uDeeFF8WW83Y_xNvuIlXDje8OB9JBRBW4MA/

We are using the 1st generation electronics board.

It does seem that the wind sensor is behaving now. We’ll be testing the project in the wild with real wind soon. Temperature and the other values also do change.

The POCS installation guide has been updated to include the fix for the camera issue, which you can try:

(source)

Thanks @tmcook!

Just to clarify, Ubuntu will try to auto-mount the cameras and treat them like a directory on the filesystem. That is because most people expect to be able to plug their camera into their computer and download their images.

The program above is part of the gnome virtual file system daemon that is always running on the computer (once you have installed gphoto2) and waits for camera to be plugged in.

This solution just makes that program non-executable so should fix the issue.

However, I think there is a better solution out there that I have used previously and will look for later today that actually just disables the daemon itself rather than makes it non-executable. The end result is sort of the same but it’s a bit cleaner to do that way.

Thanks! That did the trick …

We seem to have gotten things mostly working, but have a few questions.

How can we access the “administrative website” of PAWS? Also what is the system for uploading the images to the central server? How can we verify this is working or configure it? Also what is the process for manually uploading images if needed (as the FAQ describes for an offline system)?
Thanks in advance.

Hello! We have been progressing and testing our PANOPTES unit, but we were wondering how to upload the images. In a past thread there was a mention of an API key that lets you upload files to google, etc. Did you ever send that to us? I cannot recall…

Also in the pocs_local.yaml file the section of the Google Network looks like this…
######################## Google Network ########################################
By default all images are stored on googlecloud servers and we also
use a few google services to store metadata, communicate with servers, etc.

See $PANDIR/panoptes/utils/google/README.md for details about authentication.

Options to change:
image_storage: If images should be uploaded to Google Cloud Storage.
service_account_key: Location of the JSON service account key.
################################################################################

The directory that this README.md seems to be located at /var/panoptes/panoptes-utils/panoptes/utils/google which is different than above …

Thanks for your help!