Welcome to the new FlexRadio Community! Please review the new Community Rules and other important new Community information on the Message Board.
If you are having a problem, please refer to the product documentation or check the Help Center for known solutions.
Need technical support from FlexRadio? It's as simple as Creating a HelpDesk ticket.

A Cultural Laziness?

Dave - W6OVP
Dave - W6OVP Member ✭✭
edited June 2020 in New Ideas
From a recent IEEE article comes this interesting observation:

I believe the relative ease—not to mention the lack of tangible cost—of software updates has created a cultural laziness within the software engineering community. Moreover, because more and more of the hardware that we create is monitored and controlled by software, that cultural laziness is now creeping into hardware engineering—like building airliners. Less thought is now given to getting a design correct and simple up front because it’s so easy to fix what you didn’t get right later.

Every time a software update gets pushed to my Tesla, to the Garmin flight computers in my Cessna, to my Nest thermostat, and to the TVs in my house, I’m reminded that none of those things were complete when they left the factory—because their builders realized they didn’t have to be complete. The job could be done at any time in the future with a software update.

https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-dev...


Comments

  • Doug - K3TZR
    Doug - K3TZR Member
    edited May 2019
    Dave,

    As a retired software developer I'm very aware that more and more of life is controlled by software. Unfortunately, some of what is software controlled is also life critical (e.g. airplanes). To design a bridge you need an Engineering degree, a PE certification, etc. To design software, there's no similar set of requirements. In addition, the "triple all your calculations and nothing will ever break" thinking of civil engineering has no counterpart in software engineering. One errant line of code can "crash" the entire system.

    It's a serious problem deserving of immediate attention. As the article points out, money may be in the way at times.
  • James Del Principe
    James Del Principe Member ✭✭
    edited May 2019
    Dave, based on your examples, it would be easy to agree with you and in many respects I do. That said, however, it is tempting to confuse an upgrade or added feature, feeling that it should have been provided in the first release. The problem is the nature of software. It can always be improved with features added. Then a developer must make a decision. When should a product or SW release be rolled out?   Should it have every possible feature?  In that case it would never see a customer.  Remember that perfection is the enemy of the good.  So I really don't disagree with you but considerations must be taken.     Best 73, Jim
  • James Charlton
    James Charlton Member ✭✭
    edited January 2020
    A fundamental law of coding:
    Users = beta testers

    or, let the customers find the bugs, they paid for that right  ;-)
    Jim Charlton  AD0AB
  • Reggie
    Reggie Member ✭✭
    edited May 2019
    Just because there are certifications and standards do not mean civil engineers do not make mistakes with their bridge designs.  Last year, a newly built bridge collapsed in Miami, FL.  Medical doctors are educated for years, pass multiple certifications, and do years of residency, but they are not infallible.  They **** people every day.  Software engineering is no different.  Software engineers can run simulations, they can do alpha testing, and beta testing, but there are always bugs because no complex software program is perfect.  Look at Obamacare website.  They spend nearly $2 billions dollars creating that website, and it had major issues.  Sometimes money is not the issue.


  • James Del Principe
    James Del Principe Member ✭✭
    edited May 2019
    James, I just had to chuckle. In my past life, I spent 43 years in medical x-ray for a manufacturer.  We did alpha testing in house and beta testing for selected customers (volunteers).....still had issues over time with weird combinations of usage or even certain dates....a September 1 fault and a leap year fault......  customers were NOT amused.
  • KC0EM
    KC0EM Member ✭✭
    edited May 2019

    Perhaps, I am a bit too old to add to this discussion, but what I see as part of the problem is the lack of connection between the person doing the programming and the end user of that program. My first job was working in a postdoctoral research laboratory. There was a mixture of EEs, physicists, and other scientific professionals. One of the major problems was handling all of the data being generated in the experiments. I remember seeing high-speed printers turning out foot tall piles of paper that were nothing more than numbers. Some members of a team would want to look for patterns in the printouts. Some members would want to test theoretical equations against the printouts and so on. If you wrote a program to test a theoretical equation against the data set and it didn't work, you had to look that researcher in the eye. It was tough enough to see the disappointment when a theoretical equation didn't work out that you didn't want your work to make even a bigger mess of things. A programmer today never gets to see the end-users disappointment when their programming efforts fail or go awry.


    Footnote: I was working there in the 1970s. My biggest hit in programming was one that allowed the user to change the strategies in card counting to maximize their odds playing blackjack. I was surprised by how many scientists are gamblers.

    VC
    KCØEM
  • Doug - K3TZR
    Doug - K3TZR Member
    edited May 2019
    Reggie, I think you are correct; standards don't stop these types of failures. Nothing will ever be perfect but the software development world is the "wild west" in many businesses. In my experience, hardware companies are the worst, their view is that software is an unfortunate need to be dealt with as expeditiously as possible.

    As we are in the process of inserting software into every aspect of life I'm hoping that the 737 fiasco will be a wakeup call that some safer, more thoughtful approach is needed.
  • James Del Principe
    James Del Principe Member ✭✭
    edited May 2019
    I think you are on to something here. Too much of a disconnect between developer and end user. BTW, you are never too old to contribute. In my case, I am old on the outside but young on the inside.
  • k0eoo
    k0eoo Member ✭✭
    edited May 2019
    I disagree Dave, Every product I ever developed and shipped was working perfect, as per the original design.  Unfortunately, after my many customers got a hold of my products and ran them in ways I never anticipated, problems cropped up that needed to be fixed, some by hardware but mostly and hopefully by software...  As a designer/developer, fixing problems via software/firmware is so much cheaper/faster and better for the customer than via hardware, consequently, all my designs started going in that direction.  Big S little H....  YMMV

    Dennis, k0eoo
  • Steve Sampson
    Steve Sampson Member
    edited May 2019
    People - and their desire to make things work properly - hasn't changed.  Systems today are so very much more complicated than any historic levels (and I've run a number of developmental programs), the opportunities to make - or miss - mistakes pushes human ability.  Complexity and perfection don't play well together.   
  • Craig Williams
    Craig Williams Member ✭✭
    edited May 2019
    Resistance is futile. 
  • James Del Principe
    James Del Principe Member ✭✭
    edited May 2019
    but......capacitance is reactive.
  • James Charlton
    James Charlton Member ✭✭
    edited May 2019
    Hi James,
    Thank you.  Another fundamental rule in the software business is:  "Never buy version 1.0."

    In the interest of full disclosure, I've been both a software writer and tester, but that doesn't stop me from making fun of my fellow code-monkeys.

    I believe there is some theory that says no program can never be proven to be bug-free.  As you point out, there are so many combinations of inputs etc that it is almost impossible to test them all.  

    One method of testing was to set up a test bed and start the new code and measure how long it took for a bug to reveal itself.  At first, that Time to Failure was measured in seconds.  But as fixes were implemented it stretched out to hours and days.  So, the question wasn't is the software bug free, but how long do we test before it can be released?

    That was a management, not a technical, decision.

    I'm not trying to justify sloppy or buggy code.  Now that I'm retired and out of the business of writing and testing it I say that all code should be signed by the author so we can find them and take away their keyboards!  Better yet, their phone numbers should be published so we can call them at night and weekends then their stuff doesn't work!

    That was a lot of work, now it's time for my nap.
    73,
      Jim Charlton  AD0AB








  • Patrick
    Patrick Member ✭✭✭
    edited June 2020
    A lot that goes on in product design is driven by the marketing side of the business. I think that this was something that will be shown in the Boeing incidents. The desire to compete with the competition on price some times trumps the engineering side. In this compromising safety, with unintended or on educated sales force actions. On the engineering side trying to get the software to account for all conditions no matter how trivial has consequences. Evaluation is difficult and time consuming and can be costly the more complex the engineering. Is.
  • Erika - KØDD
    Erika - KØDD Member ✭✭✭
    edited June 2020
    I'm with Dennis K0EOO on software design verses end users quandary.

    The 737 Max Series Aircraft are NOT Tesla Autos, Nest thermostats, Garmin or Flex Radios.  The problem here is a CURRENT design of 737's the 700, 800 and 900's were confronted with AIRBUS' competing product that was smoking the Boeing in fuel economy.  When you're burning thousands of pounds in fuel during a flight, Better Gas Mileage is Important at keeping an airline's cost per seat mile as low as possible.

    Well a NEW engine, an even higher bypass turbo fan was developed and to bolt a pair of those units on to the 700-800-900 airframes Boeing needed to extend the landing gear, and shove the engines closer to the wing and further forward.

    The end result is an aircraft that additional **** causes the NOSE of the aircraft to rise increasing the angle of attack...  They had to implement this MCAS system to allow the aircraft to FEEL LIKE the previous versions of the aircraft so pilots could move between them without feeling a change.

    The airlines just need to implement differences training between the New Generation 700 etc and the Max...  The system allows the Airframe to handle LIKE the previous airplanes.  Here's the wording.

    ****The Maneuvering Characteristics Augmentation System (MCAS) is a flight control law managed by the flight control computer (FCC) and introduced on the 737 MAX to help it handle like a 737 Next Generation (NG), particularly at slow speeds and high angles of attack (AOA).****

    The problem is the dual Angle of Attack Sensors were optional even with the MCAS (they made them mandatory now... but the pilots still need training if an AOA sensor fails and how to diagnose that.) Unfortunately there were a couple aircraft that fell out of the sky due to the pilots not receiving enough simulator and diagnostics training along the way and had no clue what was happening.

    I watched an in cockpit simulation of the second crash with discussion while it was happening and I was unable to follow why the A/C pitched sharply down and everybody died...

    I stepped out of an airline pilot's seat in 1993 and admit being rusty however I'd never felt like I was out of control for no reason in perfect flying conditions before, I was able to follow their flight right up to the point the pilots started freaking out.  Of course I couldn't feel the plane fighting at the controls. DUH?

    This IS nor WAS ever a software issue...  This in both cases had been an AOA Sensor failure issue.  Even on the twin sensor systems, the pilots NEED to be trained at How to diagnose which sensor has failed and which is correct for them to properly fly the plane.  Do some of that Pilot S*** Mav...  I do believe these pilots didn't get enough training to safely fly these birds.

    These ARE NOT software controlled airplanes...  So the problem isn't bad software the problem is/was inadequate training.  Boeing is NOW adding lines of code to anticipate undertrained pilots???

    What about SDR?  Advances and revisions in SDR software if an issue was found and a fix was designed for that issue that's one thing... Adding additional features that's different, Repairing an issue with the original hardware?  Doubtful.

    Even with an SDR radio $$$$ must be spent in the original design to build components in the Signal Path like LO's, and A/D conversion devices at the absolute leading edge of technology...  and then designing software that anticipates every possible feature for each of those devices to please the customers  When You're talking new technology it would take Johnny Carson as Carnac the Magnificent  to foresee every possible vision the customer might come up with or future need so the processor can deliver at the design onset.

    Be pleased they keep updating our software....  My windows 10 Pro x64 still sees a blue screen every once in a while...  Microsoft KNOWS there's thousands of potential issues running around in these computers...  NOBODY is smart enough to design a software O/S that complex with the first revision.

    Sincerely,

    Erika KØDD
  • Patrick
    Patrick Member ✭✭✭
    edited May 2019
    I think in this case the 737 MAX was designed to have the complete package installed. Eliminating part of the package had unintended results. Yes training is a factor, and not doing proper training exasperated the problem. This is why Boeing is installing the complete package with sw updates and requiring more training to insure that there is no repeat.
  • Mark  K1LSB
    Mark K1LSB Member ✭✭
    edited May 2019
    "I think in this case the 737 MAX was designed to have the complete package installed. Eliminating part of the package had unintended results."

    Pat,

    What part of the "package" didn't Boeing install?
  • Jay Nation
    Jay Nation Member ✭✭
    edited May 2019
    Mark 

    https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings#Maneuvering_Characteristics_Augmentation_System_(MCAS) 

    https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Augmentation_System

    It's been reported that Boeing was selling an economy 737 MAX package, to some of their customers that didn't include redundant angle of attack sensors, which are included in the regular more expensive packages.

        SDRgadgets

    #FlexRadio IRC Chat

       73, Jay - NO5J

  • Mark  K1LSB
    Mark K1LSB Member ✭✭
    edited May 2019
    Thank you, Jay.  Now that you mention it, to Pat's credit I do remember now reading somewhere last week about a less-than-complete package that Boeing offered as a cost-saving option.

    According to the article in the IEEE link in the OP, there were AOA sensors on both side of the aircraft but the two flight control computers were each wired only to the AOA sensor on their respective side of the aircraft.

    Thanks again,

    Mark  K1LSB
  • Erika - KØDD
    Erika - KØDD Member ✭✭✭
    edited May 2019
    Mark I believe we're talking about the optional DUAL Angle of Attack Sensors vs a Single AOA Option for Less $$$  plus the little indicators lights on the displays stating the two sensors are NO LONGER IN AGREEMENT and HELLO its time to figure which one is really working...

    I believe any additional code they write will be an attempt to make the Troubleshooting and Notification Process easier for ALL PILOTS not just the Smart Ones with advanced diagnostic skills or the proper Simulator Training..

    No Matter what, this last crew correctly disabled the MCAS system briefly I bet in a guess or by golly move...  Maybe hoping the aircraft attitude would magically self correct for some reason I suspect. (It's not going to.  They needed to retrim the bird themselves) 

    Unfortunately when the A/C trim didn't magically self correct they flipped the thing back on and the Pitch Trim did a runaway nose down situation, resulting in a Big Hole in the Desert with innocent victims burning and tossed haphazardly around the Crater.  What Can I do at this point other than shake my head.

    In the meantime Boeing isn't shipping 737 Airplanes and are hemorrhaging dollar Signs by the billions.

    One thing I DO want to state...  Between Airbus and Boeing they've had each a share of fatal crashes on the 737 and Airbus Equivellent Unit...  737's had worn out trim **** bushings in the tails and resulting crashes, and Airbus had the verticals just plain old breaking off and crashing.  This size aircraft is not only the most common, but are flown with the most takeoff and landing cycles... Have a nice day.

    Erika DD
    
  • Jay Nation
    Jay Nation Member ✭✭
    edited May 2019
    Mark 

    This economy package also eliminated, an optional pilot training program designed to train 737 MAX pilots about handling situations where the MCAS system malfunctioned. 

         SDRgadgets

    #FlexRadio IRC Chat

       73, Jay - NO5J

Leave a Comment

Rich Text Editor. To edit a paragraph's style, hit tab to get to the paragraph menu. From there you will be able to pick one style. Nothing defaults to paragraph. An inline formatting menu will show up when you select text. Hit tab to get into that menu. Some elements, such as rich link embeds, images, loading indicators, and error messages may get inserted into the editor. You may navigate to these using the arrow keys inside of the editor and delete them with the delete or backspace key.