Welcome to the new FlexRadio Community! Please review the new Community Rules and other important new Community information on the Message Board.
If you are having a problem, please refer to the product documentation or check the Help Center for known solutions.
Need technical support from FlexRadio? It's as simple as Creating a HelpDesk ticket.

Is a more structured Alpha/Beta program needed?

1235

Answers

  •   VE6KWA
    VE6KWA Member ✭✭
    edited July 2016
    Have you considered Decaf Burt ??  B) 
  • KC9NRN
    KC9NRN Member
    edited October 2016
    I'd be more than happy to fix your Windows 10 issue. :) I can't imagine having an issue crop up that bricks a device and then you get all the "experts" telling you how to fix it. The network I manage runs smoothly most of the time then suddenly one of the profiles on Terminal Server decides not to load the profile fully affecting several users who all have an idea on how to fix the problem even though they know nothing about Terminal Server or servers or heck even their computers but everybody is an expert on telling people what to do when they don't have to back it up. It amazes me to no end and the constant calls slow down the process of fixing the issue. I digress. I'm happy with my 6300 thus far and will be persistent on getting onto the Beta list if possible, not because I think I know everything. The exact opposite is true, hopefully the newness I bring to SDR radios can be a benefit if I know the outline if what FlexRadio is looking for during testing.
  • Jim Best
    Jim Best Member ✭✭
    edited December 2016
    Where this thread is concerned, I have enjoyed using the delete button. After a point 
    this becomes a waste of bandwidth.
  • Burt Fisher
    Burt Fisher Member ✭✭
    edited August 2016
    Ok Gerald you did all one could reasonably do.
    As far as my earlier comments:
    https://www.youtube.com/watch?v=V3FnpaWQJO0
  • KY6LA_Howard
    KY6LA_Howard Member ✭✭✭
    edited July 2016
    Gerald '' Had 2 iPads bricked so far. Neither ever totally recovered. Still glitchy. Gave them to grandkids. So it happens to the best of them.
  • KC9NRN
    KC9NRN Member
    edited October 2016
    Ok Burt, that had me laughing!!
  • Burt Fisher
    Burt Fisher Member ✭✭
    edited August 2016
    I wasn't the only one that was wrong, why am I the only one admitting it?
  • Kevin Va3KGS
    Kevin Va3KGS Member ✭✭
    edited July 2016

    Yes Burt, your right, this community has the technical base and entertainment.  That's why I (we) keep coming back here ever day. My Radio(s) are always on for stimulation while surfing through the community forum.

    Have a great day

    Kevin, Va3KGS

  • Philip KA4KOE
    Philip KA4KOE Member ✭✭
    edited July 2016
    Pull the pin and THEN release the spoon. Then start counting. Remember the Holy Hand Grenade of Antioch??? From Brother Maynard?

    https://www.youtube.com/watch?v=xOrgLj9lOwk

  • Philip KA4KOE
    Philip KA4KOE Member ✭✭
    edited July 2016
    When Neil got wrapped up in politics, etc., I lost all respect for him.
  • Philip KA4KOE
    Philip KA4KOE Member ✭✭
    edited July 2016
    In my case, they are indeed doing this very thing!
  • Ernest
    Ernest Member ✭✭
    edited July 2016
    Steven,
    Why aren't you working at Flex?
    Since you and your other collaborators here seem to have the answer to (all) the problems.
    Do you honestly feel or think that these bugs you are alleging; were set purposely? Criticising the Company and the Alpha and Beta testers for not finding these problems, clearly show your lack of appreciation for their efforts and time. 
    As others have said, ", one has to wonder if a lot of these are not caused by the operators not following directions."
    The funny part of all this, is that when I check other manufactures radio sites, (if they bother to have one) you don't hear a peep from these individuals there.
  • Jay Nation
    Jay Nation Member ✭✭
    edited August 2016
    I was thinking that once I was clear I'd just yell ...
    OK! you can put it down now and step back slowly! image
    Next thing I know, someone taps me on the back, hands me a grenade, and sez ...
    Huh? I couldn't hear what you said. 

    Run Away!!!

    73, Jay - NO5J
  • Simon Lewis
    Simon Lewis Member ✭✭
    edited July 2016
    just don't count to 5

  • Paul Christensen, W9AC
    Paul Christensen, W9AC Member ✭✭
    edited July 2019

    To Gerald and Team:

    Perhaps its worth re-evaluating the definitions being used for new releases.  I suggest keeping alpha within the 50 or so testers.  Once there's consensus between Flex and the testers, release it to the public only as beta -- and not an official release.  We in the Flex Community then have the option of sticking with an existing, stable release, or try the beta and help move it from beta to the next official release.  IOW, no new release is an official release.  Once beta and its quick fixes are made stable, it then moves to official release status.  This redefines the expectation among users (i.e., "it's beta and still in development...don't expect perfection from this release").  If anyone is in any way hesitant about possible imperfections in the beta release, then wait.  Wait for it to change from beta into the official release.   

    The problem as I see it, is the assumption that every new SSDR release is completely stable.  We all download the latest release and then get these knee-**** reactions from users after every single new, public release -- every one of them.   Why not keep the public release experimental, and state it as such by calling it beta? 

    What I've just described is Elecraft's release process.  In the early years of the K3, these same issues and annoyances proliferated the messages boards.  It quickly got out of control.  Subsequent to the change, their release process has been in place for many years and it works well (albeit not perfect) among their user base.

    Paul, W9AC 




     



  • Varistor
    Varistor Member ✭✭
    edited July 2016
    The question of how many testers are testing the product is not that interesting. The real question is who and how designs the tests plans, how are the communicated to the testers, and how do the testers follow them. If testers are simply told "Go ahead, test" the overall success and quality of the testing process will be very low.

    Test and security planning starts the minute you design (not write!) your system. IBM did a study that fixing a bug during the development process is 80 times cheaper than once the software has been released.
  • KC9NRN
    KC9NRN Member
    edited October 2016
    100% agreed Philip.
  • KC9NRN
    KC9NRN Member
    edited October 2016
    A fresh set of eyes is never a bad thing, some of the reasons for not taking on new beta testers make zero sense to me but heck, I'm just a network engineer, what do I know.
  • John-K3MA
    John-K3MA Member
    edited December 2016
    Burt, whom else do you believe was wrong and why?  I am not sure I understand why you even think you were wrong?
  • Jay Nation
    Jay Nation Member ✭✭
    edited August 2016
    Until I'm given the chance to prove my competency, I accept the fact that I'm judged less competent. No one will know the real level of my competency until I'm given the opportunity to demonstrate just how competent, I really might be. The same classification of "less competent" applies to everyone else being underutilized.  Call me names, or watch me work. I don't understand why so many fingers need to pointed at the alpha/beta testers, before any real cause of the problem is known. I'd rather spend time with "Team Less Competent", "Team Finger Pointing" isn't likely to be any more competent, and might be a lot less likely to volunteer to do any of the work. Who wants to be the one being pointed at the next time something fails to fail during testing. The current crop of alpha/beta testers got us all this far. They might have a clue about what really went wrong, and how to avoid a repeat down the road.

    So did everyone enjoy seeing this Firework display here in the Community?

    73, Jay - NO5J 
  • John-K3MA
    John-K3MA Member
    edited April 2019
    Gerald thank you for your official response regarding the Maestro issue.  My comments were more based upon the total situation in release 1.8 of which I understand the Maestro is of major concern.  Would you also care to address the other item which has been a issue for many regarding the power foldback problems people are having with the release?  I did not see that mentioned specifically in your response above.

    Also, I have personally heard one Beta tester mention that "they were not aware of the power foldback change prior to the release".  I am not sure if that comment applies to only him or if Flex wanted him as a beta tester to test the power foldback functionality or not.  All I can be certain of from his statement is that he is a beta tester and did not beta test the power foldback functionality.  I only mention it here so that if you wish you could cover the breath of the testing of the power foldback in your response.
  • K7NXT
    K7NXT Member
    edited November 2017
    I think the reason that this thread refuses to die is because there are two diametrically opposed viewpoints about the embedded and network software.  

    But first I would ask the group - would it be different if we were discussing software that runs your car?  Or is Ham radio a special case?  So here are the two viewpoints that I see:

    Viewpoint 1. Ham as a hobby is all about experimenting, tweaking, and getting involved in technology. Therefore, bring on the incremental releases, play with them, learn about networking, WiFI, VPNs, etc. and revel in the technology including the problems and solutions.

    Viewpoint 2. People are willing to pay premium prices for premium products - products that just work. Once manufacturers started putting embedded computers into their products and rely on networks, they also take responsibility for the complexity that results.  

    I see that the discussion about the number and types of testers needed, the labels given to people who disagree, etc. as evidence of these two viewpoints.


    Many of us waited for months to get a Flex on the premise  that the product would be solid when we received it.  I think most of us still have the patience for Flex to work out problems that exist, and because of that, I concur with the idea of fewer and more robust releases.

    73s.







  • EA4GLI
    EA4GLI Member ✭✭✭
    edited November 2016
    Paul, that is an excellent suggestion and I hope FRS seriously considers it.
  • Gerald-K5SDR
    Gerald-K5SDR FlexRadio Employee ✭✭
    edited December 2016
    Paul, I truly appreciate the suggestion. However, we have been doing multiple software releases per year as a company for over 13 years now. In the early days we did public beta on every release just as you suggested and it was a total support nightmare because people don't properly self select in or out and the world thinks you are perpetually in beta. When we switched to a private beta on PowerSDR believe me our overall quality of life improved. In a perfect world where everyone follows the rules as you proposed it would be great. Public beta did not work well in our experience for the reasons stated. Remember that with all their infinite testing resources and billions of customers, both Apple and Microsoft bricked two of my devices during OS upgrade respectively? The latter is still bricked. If a frog had wings he wouldn't bump his but. ;>)
  • Gerald-K5SDR
    Gerald-K5SDR FlexRadio Employee ✭✭
    edited December 2016
    We provide release notes and a shared testing spreadsheet by issue for every single beta release. I don't know who the tester is but he must not have paid attention. If the test team had reported a problem with this feature, we would have held the release or removed the feature. 100% of the testers had the software and it requires no special expertise to know if it works or not.
  • Burt Fisher
    Burt Fisher Member ✭✭
    edited August 2016
    Others were saying Flex needed to diversify their testers and Gerald said they already had (this among other things).
  • KC9NRN
    KC9NRN Member
    edited October 2016
    Yet it works for Elecraft, EVGA, NVIDIA, AMD, Intel, Microsoft, Apple and a large assortment of other companies too long to list here. The thing is Paul is spot on, it's not just Elecraft who does this, heck I have a beta driver for NVIDIA sitting in my system tray right now, I can choose to install or not and I accept the risk for what happens.

    I am on the fast ring on the Windows 10 Insider program, it too is prompting me to install the latest Windows 10 build, again I take the risk, not Microsoft even if bad news does affect them, they can say I accepted the risk of using Alpha or Beta software, something I have done for 25 years, I know I'm not alone in this community that is willing to accept the risk, you can't **** if you accept the agreement, maybe you need to change your agreement.

    You upgrading two devices and now they're bricked from Apple AND Microsoft is so far from the norm you should play the lottery that's how crazy the odds are and the Windows 10 machine, is NOT bricked. It can be formatted and all is well, the Maestros that have to be returned to be fixed, not so much, maybe you format them but for the user, they are truly bricked.

    The thing is when performing an upgrade on top of an OS that has had how many things installed over time, you shouldn't be shocked when bad things happen and by accepting Microsoft's terms them there is the breaks.

    When we install new versions of your software are we supposed to perform an upgrade or uninstall the previous version first? 50 people testing software is too few, maybe for Alpha its fine and I'm sure for 13 years maybe it was fine but the hardware you're selling is no longer a client, it's server class and 50 people beta testing for not only the 6000 but also the Maestro, is ridiculously small unless of course your user base is just a few hundred which its not.

    I am spending $3,800.00 or so for the 6300 and Maestro, compared to those who bought the 6700 and Maestro I'm small potatoes but to me that $3,800 represents a lot of saving so hearing that the beta user size is 50 people, shocked me, I thought Flex had more resources than that to support a larger beta group.

    Perhaps a re-think is in order considering you added a another piece of hardware that has the promise of WAN, is a head unit for your signature line of transceivers and has the same amount of testers that were there before the Maestro.

    How many of the 50 testers purchased Maestros? If all 50 do you really think that was enough of a sample size to vet all possibilities? Rhetorical question.

    I am not writing this to **** on your parade or to say you're wrong, not even a little but I do think that your response claiming well Apple and Microsoft bricked my stuff so how we're doing things is fine, a public beta works for hundreds of companies, maybe you could re-evaluate? 
  • Bill -VA3WTB
    Bill -VA3WTB Member ✭✭✭
    edited December 2016
    I like what Gerald said the other day, Each of the testers could have upgraded hundreds of times without seeing any problems. And his point, that this could happen to any software company, and as we see, it does, not only a Flex problem.
  • KC9NRN
    KC9NRN Member
    edited October 2016
    Bill,

    The amount of testers using the same hardware and OS and not changing them out means that likely yes, they can upgrade a few hundred times and get similar results.

    The facts are, the more who actually test it, on different systems the more results you'll get that are valid. Unless each beta tester changed OS, hardware or heck the network switch or something like that results are easy to duplicate.

    Maybe I'm not seeing it the same way as Gerald is, maybe I have been involved in too many beta tests where fewer meant less valid results thus they added testers to get more valid results or at least more results from differing systems to say the results given were valid.

    I love the hardware and the software, I just don't agree on the method of using so few people to test and validate them.

Leave a Comment

Rich Text Editor. To edit a paragraph's style, hit tab to get to the paragraph menu. From there you will be able to pick one style. Nothing defaults to paragraph. An inline formatting menu will show up when you select text. Hit tab to get into that menu. Some elements, such as rich link embeds, images, loading indicators, and error messages may get inserted into the editor. You may navigate to these using the arrow keys inside of the editor and delete them with the delete or backspace key.