|
|
Post by kenneth on Jun 3, 2023 19:40:57 GMT -5
Good Afternoon All,
I'm a Weather Observer with a programming hobby and I had an idea that could make our jobs just a wee bit easier. Basically, on the beta aviation weather site(soon to be the main site), I wanted to create a script to fulfill numerous of our tasks automatically. There's no replacing humans when it comes to checking for errors, but this does a good job so far. It also allows for permanent storage of reports, and allows for searching up to the past 360 hours of reports from any station.
There's a lot more stuff in between, but I just wanted to ask my fellow observers within the 50 states and Puerto Rico what you think of this, how useful this could be to your station, and more.
Due to the limit on the length of an attachment, I am unable to attach the video of the script in use directly to this post, so here is an unlisted YouTube link of it:
Some potentially interesting information I've gathered with regular use of this script & of the script itself:
-CWO Stations(135 total) make an average of 15-25 detected errors a day, combined.
-You can store up to approximately 56,250 reports(Numerous years of reports for one station)
-There is error detection for every section of the report
-The script is currently 3,995 lines of code.
-57 different types of errors are currently detectable.
-Limited access, contact me if you want access to it.
There is one big hurdle though - regional & site-specific regulations & rules. This limits a lot of what I can do and does require some tuning of the error detector settings to make this script work for stations across the US. One big problem is, I don't know where exactly online I can find information about these specific regulations. If anyone could give me some pointers it would be a great help!
I don't intend on selling this script or making money from it, I originally made the script for myself to make my job a little easier(hence the bad design), and eventually started to develop it for general use. Does anyone think this script is useful? Does anyone think it could potentially be useful? If so, why?
I look forward to hearing any and all responses, if I get any. I just want to know if I should really go public with it or just keep it to KYNG.
Thanks!
P.S. -- if this script does go public, this will just be for CWO's, not ATC!
|
|
|
|
Post by movedsouth on Jun 25, 2023 7:16:28 GMT -5
Thanks Kenneth, I appreciate the thought and effort. I do think there is a huge amount of station specific stuff that will drive you and your program crazy. Station specifics are NECESSARY, because conditions at the various stations vary. What some would mark as an error are routine at others.
|
|
|
|
Post by kenneth on Jun 25, 2023 18:25:57 GMT -5
Thanks Kenneth, I appreciate the thought and effort. I do think there is a huge amount of station specific stuff that will drive you and your program crazy. Station specifics are NECESSARY, because conditions at the various stations vary. What some would mark as an error are routine at others. I realize that station specific stuff is necessary, which does greatly complicate matters. While there are station specific/region specific rules and regulations, there are rules that apply all throughout the United States, and this program is able to detect the majority of those common errors. When starting up the script, the user will be able to customize the error detector settings. I made these error detector settings so that this script can be used by stations across the United States. For example, you can modify the highest base height for a layer containing significant clouds, as that can vary by station. To state the obvious, this means that a sizeable portion of potential errors committed by other stations won't be detected by the user using the script, but there is still decent error detection. You set your "Home Station" when you first start the program and that is the station that has the customized error detection. Running this script for the past month while I've been developing it, I've noticed typos consist of the most common errors. I've seen the Altimeter start with "A20" instead of "A29". Furthermore, many errors get detected for missing wind/visibility. If you know of any online sources that specifically list regulations for any given region, I would greatly appreciate if you could provide the link. By the way, just so you know what types of general errors are detected by this script, I've been running this error detection on all CWO stations off and on for the past month. I will list some of the saved errors below: Missing or invalid wind section. PATA 080152Z VRB003KT 10SM FEW060CB BKN100 23/07 A2964 RMK VIRGA N TCU NW-NE Missing or invalid wind section. KDAL 050653Z COR 114/10KT 10SM TS FEW018 SCT030CB BKN070 BKN250 23/17 A2992 RMK AO2 PK WND 11027/0653 TSB49 PRESRR SLP123 OCNL LTGIC N-E-SE TS N-E-SE MOV NW T02280172 Missing or invalid wind section. PAFA 030153Z COR 270/04KT 10SM SCT038 BKN046 OVC120 09/00 A2989 Extremely low and unlikely altimeter entry. PFYU 030151Z 22009KT 10SM BKN050 09/ A2084 OVC053 cannot end with "3". Must be "0" or "5". KPVD 171751Z 15005KT 10SM FEW017 BKN043 OVC053 18/14 A2959 RMK AO2 LTG DSNT ALQDS SLP019 60017 T01780144 10189 20150 56011 $ SCT125 ends in "5" instead of "0". KABQ 132052Z 32010G16KT 10SM SCT125 27/M07 A2998 RMK AO2 PK WND 26027/2003 SLP078 T02671067 57021 OVC052 cannot end with "2". Must be "0" or "5". KDAL 140053Z 06005KT 5SM BR BKN039 OVC052 26/24 A2974 RMK AO2 SLP064 T02560244 By the way, I saved the best for last. Layer height in descending order. KSHV 180156Z 33007KT 10SM FEW036 BKN140 BKN130 27/23 A2981 RMK SLPNO What do you think? |
|
|
|
Post by movedsouth on Jun 27, 2023 8:46:41 GMT -5
7900.5_ is what it is, and so are the contraction regs, the NWS handbooks and more - these might be workable into your program. Your script might be helpful, but it is not and cannot be a substitute for human QC - and I fear that it will both catch "errors" that are not and miss errors that are, and that someone might think that this latest attempt at automating human judgment should take the place of humans. Properly used and augmented, it might save us some time. Forgive me for being skeptically old-school - I do appreciate the thought and effort you've put into this - but I'm a tough sell. IF you can allow each station to add or modify parameters to cover for station idiosyncracies, you might yet be able to sell me. We're doing training and QC just fine right now without it.
|
|
|
|
Post by kenneth on Jun 28, 2023 0:52:33 GMT -5
7900.5_ is what it is, and so are the contraction regs, the NWS handbooks and more - these might be workable into your program. Your script might be helpful, but it is not and cannot be a substitute for human QC - and I fear that it will both catch "errors" that are not and miss errors that are, and that someone might think that this latest attempt at automating human judgment should take the place of humans. Properly used and augmented, it might save us some time. Forgive me for being skeptically old-school - I do appreciate the thought and effort you've put into this - but I'm a tough sell. IF you can allow each station to add or modify parameters to cover for station idiosyncracies, you might yet be able to sell me. We're doing training and QC just fine right now without it. I appreciate your input and will definitely take your opinion into account. One thing I would like to reiterate is that I am not attempting to sell this program, instead giving it away for free to CWOs only. I just want to keep this script to CWOs and that's that. The reason for that is because I noticed many foolish mistakes that numerous sites make that us all look bad & adds fuel to the fire. Perhaps this script wouldn't be of as much use to a station who rarely commits errors, but those stations are few and far between. I would like to note that this script is not intended just to catch errors in retrospect, but also to catch errors on reports oneself is about to disseminate. I am definitely applying the general regulations set forth in the 7900.5 and similar documents, but there are some region-specific regulations that are not noted anywhere that I know. You are definitely correct that it will detect some things as errors that most certainly aren't, which is why I've been testing the error detection(automatically) around 4,900 times a week. Over the past month and a half or so, I have found numerous things being incorrectly detected as an error, which I have rectified. I intend this script to be more of an assistant than anything, and to be particularly useful to stations that commit over 2-3 errors a month. I have a section of the settings called the "error detector" settings. These settings can be tweaked to toggle certain elements of the error detection and to set certain values that apply to your station specifically. For example, at KYNG the highest base height for a low-level significant cloud permissible is 6,500 feet. In this script, I set the max base height at 6,500 feet and so, if the program detects a report from KYNG here including "BKN070CB" for example, it'll detect that as an error. It sounds like your station is very diligent, which is very good. However, many people in this field from either lack of knowledge/experience or lack of care aren't so diligent. These individuals are the ones that typically make these dumb errors that make us all look bad. I am doing very, very thorough research though and I will continue to test every single element of my error detection as I continue to expand it's capabilities. |
|
|
|
Post by movedsouth on Jun 28, 2023 9:22:15 GMT -5
Kenneth, I appreciate your intent, though skeptical of the result - especially if it auto-corrects what it detects as an error, regardless of the local accuracy of the error index. It is only a short step from identifying error and auto-correcting. This could be used by the FAA as a weapon against our program, if it got into the wrong hands. Regarding diligence: Any fool in the CWO program that is too laid back or otherwise fails to take what we do seriously as a matter of aviation safety critical to the NAS needs to resign immediately. What we do is too critical to allow for "human error" - I may be a bit zealous, but the muckity-mucks who make decisions from cubicles without windows and do not fly in airplanes watch every error we make and use it to further the argument that our program isn't worth the money and that ASOS can do better. That said, we can still enjoy our work and make our workspace most pleasant - if we all maintain great attitudes and encourage each other - this said after 27 years in the program, 25 as an SWO. Could your script be tweaked for station specifics locally? - Make it locally programmably user-friendly as possible!
|
|
|
|
Post by kenneth on Jun 28, 2023 16:49:35 GMT -5
Kenneth, I appreciate your intent, though skeptical of the result - especially if it auto-corrects what it detects as an error, regardless of the local accuracy of the error index. It is only a short step from identifying error and auto-correcting. This could be used by the FAA as a weapon against our program, if it got into the wrong hands. Regarding diligence: Any fool in the CWO program that is too laid back or otherwise fails to take what we do seriously as a matter of aviation safety critical to the NAS needs to resign immediately. What we do is too critical to allow for "human error" - I may be a bit zealous, but the muckity-mucks who make decisions from cubicles without windows and do not fly in airplanes watch every error we make and use it to further the argument that our program isn't worth the money and that ASOS can do better. That said, we can still enjoy our work and make our workspace most pleasant - if we all maintain great attitudes and encourage each other - this said after 27 years in the program, 25 as an SWO. Could your script be tweaked for station specifics locally? - Make it locally programmably user-friendly as possible!
This script doesn't make an effort to "auto-correct" anything, it merely tells you what error is present in the report. An observer worth their salt will be able to discern whether there actually an error or not. That being said, I have tested this script for a while and in it's current state, I can't remember the last time that something that wasn't an error was detected as one. This script is strictly my intellectual property. I am the sole distributor of the script and it would be illegal for the script to be used in a way that I did not intend or for it to be redistributed without my explicit permission. I will, to the best of my ability, ensure that this script does not get into the wrong hands and that any unlawful use or distribution of my script will be dealt with by all means. From viewing reports from around the country, I can assure you that, unfortunately, there appear to be many observers who sit back and collect a paycheck. I won't say which site, but today during these smokey conditions a CWO site neighboring ours included MIFG with a 6°C temp/dew point spread and nothing else with a visibility of 1SM. Nothing else restricting vis. This continued for around 5-6 hours straight. Yes, the user can tweak the error detector settings for their Home Station. For example, I can tweak the error detector settings for my home station that I set in the program, KYNG, and all reports from KYNG will have the advanced, custom error detection. I understand that many folks in this field are not tech savvy, so I'm trying my best to make this script as easy to use and straightforward as possible. |
|
|
|
Post by movedsouth on Jul 2, 2023 23:28:06 GMT -5
Respectfully, I appreciate the effort and intent - it is not you and what you have written, but what others might do with your idea that concerns me more. Much like what Microsoft did to Apple when they reverse engineered the MacOS to produce the "original" Windows. That your script works well for you, great. That it flags what it is programmed to call an error and then have a station SWO have to fix the programming to fit their station's need just sounds like more work than we are already doing and more than is needed. Enemies of the CWO program will use it or an avatar to our detriment. Discipline slackers, yes. Bottom line - no thanks, I'll pass on your script.
|
|
|
|
Post by kenneth on Jul 4, 2023 16:04:52 GMT -5
Respectfully, I appreciate the effort and intent - it is not you and what you have written, but what others might do with your idea that concerns me more. Much like what Microsoft did to Apple when they reverse engineered the MacOS to produce the "original" Windows. That your script works well for you, great. That it flags what it is programmed to call an error and then have a station SWO have to fix the programming to fit their station's need just sounds like more work than we are already doing and more than is needed. Enemies of the CWO program will use it or an avatar to our detriment. Discipline slackers, yes. Bottom line - no thanks, I'll pass on your script. I understand your concerns and respect your position. This script isn't for everyone. Your skepticism is certainly warranted considering what the FAA would do with the CWO program if they could. Thanks again for your input and while I am going to continue developing this script, I will factor in your opinions and do my best to never allow this script to be weaponized against our program. Thanks Again! |
|
