Data Tools
In support of getting quality data into PTAGIS and allowing users to retrieve and make use of those data, we have developed several tools for both data contributors and data users.
Data contributors can use PTAGIS-developed software packages to collect, manage, format, and submit both MRR and interrogation data. When those data are submitted for inclusion in the PTAGIS database, automated procedures validate the data and notify data contributors about any potential issues and about the final status of the data submission.
Data users can query and retrieve data using a set of online reporting tools or download the raw data files through this website.
Software
Tagging software P4 is used to collect associated data when fish are marked or recaptured. Tag readers and other devices (such as balances) can send tag codes and other data directly to P4 to prevent transcription errors. P4 provides a customizable data entry interface and can be operated with a digitizer tablet for fast data entry.
Interrogation software M5 is designed to run unattended on a computer and provide continuous recording of observation data in real-time, and automated submissions to PTAGIS. M5 is used at the sites maintained by PTAGIS and provides separation by code functionality. It can also be run on a low power Linux system, such as a Raspberry Pi, to provide real-time data collection at remote sites.
Organizations operating interrogation sites without dedicated data collection computers can use I5 to import and transform raw transceiver data into properly formatted files for submission to PTAGIS.
Data Processing
- Verify the user is authorized to submit data
- Verify the file format is correct
- Verify the data meets all validation rules
- Check for duplicate records
If the file passes all validation checks, the data contained within the file is loaded into the transactional database and the data file is archived in perpetuity. If a file fails any of the first three validation checks, it is rejected and the data is not loaded into the database. If a record in a file is a duplicate of an existing record, the file is loaded, but that record is marked as a duplicate. Upon completion of file processing, the data contributor is notified about the status of their file submission.
New or updated data loaded into the transactional database are pushed to the reporting system database once all files in the queue are processed. Data is generally available to query from the reporting system within 30 minutes of being received by PTAGIS.
Query
PTAGIS data are available to view and download by anyone via this website. Data users should read and understand the Data Policy, and the of the PTAGIS Data Specification prior to querying the PTAGIS dataset.
PTAGIS data can be queried and downloaded in a number of ways:
- Several reports provide a limited view of specific data without the need to login: Record Summaries displays summaries of all records in PTAGIS. Complete Tag History displays all events recorded for up to 50 tags. Observations displays tags detected at a single interrogation site and Adult Ladders displays tags detected at one or all of the fish ladders at a dam
- Advanced Reporting tools allow a registered user to build a custom query, export the results in a variety of formats, save the query definition, and schedule the query to run periodically
- A RESTful API allows for limited system-to-system data transfers
- Data extract files allow those with dedicated database management systems and personnel to build and maintain a mirror copy of the PTAGIS reporting system database (contact us for more information)
- Raw data files are stored in perpetuity in the PTAGIS file archive and can be viewed or downloaded via the dashboard