1.4 Suggestion

  • 3
  • Idea
  • Updated 3 years ago
Since we now know that 1.4 needs to bake a little longer, I was thinking about ways to smooth the process. This version is probably the largest undertaking since SmartSDR itself and was never likely to fit into the schedule for point releases. As a result, I was wondering if it wouldn't be prudent to change how it is released.

Flex has a capable group of alpha testers and I'm sure they represent the most technically competent of current users. Unfortunately, they don't always see things from the perspective of average users. Given the complexity of 1.4, maybe it would be better to give testing another phase. Once 1.4 is working reliably with the alpha testers, release it to a beta with the general population. This would allow users who don't have to have the latest and greatest or are less "cutting edge" the comfort to sit it out and wait for the final release. This would hopefully prevent a surge of support issues and generally make the release of this major update easier to manage. It might also make it easier for management to get it out the door by removing the pressure to chase down every single issue or optimize every line of code.

I know I would sit on the side for a while to see how things are going. I would love to be reading about beta users experiences with 1.4 instead of all the speculation and frustration that is filling the forum of late. Once things were settled with 1.4 I would probably upgrade and get ready for what I'm really waiting for...1.5.

Jon...kf2e
Photo of Jon - KF2E

Jon - KF2E

  • 629 Posts
  • 188 Reply Likes

Posted 3 years ago

  • 3
Photo of Brent Parker

Brent Parker

  • 93 Posts
  • 12 Reply Likes

Jon - That actually not a bad idea, IMHO.

Many of the video driver (Nvidia) sites have posted their current "stable" release, but if you want to try the bleading edge beta, you can. They don't support it in terms of giving response, but certaintly take input.

I've upgraded and downgraded smartsdr several times. It takes a few minutes and is pretty simple, but you need to know a little about what you are doing. For those not used to the un-install process, they might not want to be out there on the bleading edge.

Example, on my work computers, where production is important, I only have "stable" releases. Ocassionally at home if I feel like expermenting, I give beta a try.

So as a flex user, if he just want to operate, I might be better with the stable release. If you want to experiment (in the true ham fashion) then I might jump out there!

There is certaintly a distinction between Alpha and Beta. If you are a Alpha tester, flex is depending on you to test and report back promptly. If you are a Beta user, there is less dependency on you by the company, but as a "enthusiast" you will report discovered anomilities. However as a beta release the company is not obligated to waste time "supporting" issues.

Brent  NB4AP

 

Photo of Jay / NO5J

Jay / NO5J

  • 1475 Posts
  • 220 Reply Likes
Big secret folks, It's all beta, beta all the way down. it will be beta right up until it's final. It will be final just before it becomes historical. Then we move on to SmartSDR v2.0 beta. We still haven't seen PowerSDR v2.7.3 beta. So we have to wait for that too. Your born, you wait, you die. Enjoy the waiting it's all we've got, It's what we do. Besides whining. Whining won't reduce the wait. It entertains some, annoys many. When you stop whining we know your wait is over. Get all the wait you can. 
Photo of Gerald - K5SDR

Gerald - K5SDR, Employee

  • 794 Posts
  • 1436 Reply Likes
Jon and Brent,

Thanks for your thoughts.  You are correct that this release is one of the largest undertakings since the beginning of SmartSDR.  LAN remote is the basis for WAN remote and requires very sophisticated system level integration to make it work right for multiple receivers/panadapters/waterfalls.  All this happens in a real time, client/server, multi-threaded, five processor signal processing system over an unknown network.

In many ways I like the idea of beta releases but I also have some bad experience form the earlier days with PowerSDR.  We did exactly what you were suggesting on PowerSDR until we got burned so many times by customers publicly grumbling about problems with beta software.  It created the perception with potential customers that all our software was in beta, which was not true.  We eventually quit releasing beta on PowerSDR and went to the process we have now for that very reason.  

Frankly, I would love to do as you suggest but I don't know if the general ham population is sophisticated enough to understand the software development process.  I would love to be proven wrong.

73,
Gerald
Photo of Jay / NO5J

Jay / NO5J

  • 1475 Posts
  • 220 Reply Likes
Ken
Could you perhaps map out a procedure so that other special people could earn they're own special status. I realize theres nothing fair about being special. There is another, special line. Theres is nobody in it so its a lot shorter. If you go stand in it we can all observe what it's like to be special. Thanks for volunteering. We can't hold your place in this line though. You can stand with us, or go be special. Your choice.  
(Edited)
Photo of Gerald - K5SDR

Gerald - K5SDR, Employee

  • 794 Posts
  • 1436 Reply Likes
Ken,

You said, 

Perhaps there could even be a "closed" or "non-public" section on the community for the beta testing group to help each other and offer feedback to the developers...with the proviso that this is outside the regular "Customer service" loop and that in many ways we would be "on our own."
This is probably the only way I would be open to setting up a beta testing group.  It would be counter productive for beta to add to our customer service workload or to confuse potential customers with beta related issues.  I do like the idea of gaining a broader test base than we have with the external alpha team.  We would probably want some form of written agreement to join.

None of this is a commitment - just thinking out loud in public.
Photo of Jon - KF2E

Jon - KF2E

  • 629 Posts
  • 188 Reply Likes
Gerald,

If you do decide to do this I would gladly sign up. Recently retired, I spend a large portion of my time with my radio.

Jon...kf2e
Photo of Ken - NM9P

Ken - NM9P, Elmer

  • 3916 Posts
  • 1189 Reply Likes
Gerald, I understand about the "thinking out loud" part.  
I have gotten stung a few time in board meetings while doing that!

If you get to the point of actually implementing it, please sign me up!

It think it could go along with the spirit in the video of your presentation that Howard recently posted . . .about the two different markets in the SDR field - the bleeding edge experimenters, and the high performance operators.  I find I am putting down roots on both sides of the "chasm."  The problem comes when people forget which level of support they should expect from each side of the chasm.   

One of my biggest kicks in SDR was being asked to help test some of Steve's renditions of APF and other CW functions along with NN4ZZ and others in the pre-release and early post-1.0 release  days.  I don't know how helpful I actually was, but it was fun.  I have much more time in the seat on the 6500 now and might be able to be add more to the evaluation now!

Again... I certainly wouldn't want any of this to hinder current development efforts.  That wouldn't be "SmartSDR" policy!
(Edited)
Photo of Burt Fisher

Burt Fisher

  • 1023 Posts
  • 365 Reply Likes
I would sign up and you should stipulate anyone who signs up make no statement to anyone except a Flex employee.
Photo of IW7DMH, Enzo

IW7DMH, Enzo

  • 353 Posts
  • 84 Reply Likes
Dear Mr. Gerald,

as a software developer I agree with you, but as a user of SSDR I think that the release period between two versions is a bit long.
I think also, that, after the 1.4 SSDR version, it could be helpful reviewing the process you release new software.
I would like you stop using the traditional setup process, that often require uninstalling the previous version, and that require users to become very skilled os users. (we all know Windows uninstaller is not a perfect machine).
I would like you adopt a lighter setup module, to be installed only once, and being able to deal itself whit SSDR updates. Just as it already happens with the rig firmware.
In this way developers could release, even daily, small software updates eliminating long waiting periods and requiring little skill and effort from users.
As you use to say these are "just my two cents" :)
Anyway I like too much my new Flex and I thank you very much for giving us constant updates about your products.

73' Enzo
iw7dmh
Photo of Ken - NM9P

Ken - NM9P, Elmer

  • 3916 Posts
  • 1189 Reply Likes
Hi Enzo,
While this might be desirable, it might not be possible, since SSDR incorporates not only what is in the windows program, but it also installs a new firmware image to the 6000 rig itself.  It might not be possible to do "rolling updates" in something this complex.

Ken - NM9P
Photo of Gerald - K5SDR

Gerald - K5SDR, Employee

  • 794 Posts
  • 1436 Reply Likes
Enzo,

What kind of release cycle are you suggesting?  There is a lot of work that goes into a public release.  Shorter cycles means that the percentage of overhead goes up.  Even doing a weekly alpha release cycle adds engineering overhead.  

By the way, we do real time builds.  Many times from home I grab the latest develop release at home over the VPN and run it.  I usually grab the last one of the day, which includes all of the commits.  I keep all of the older versions installed so that I can go back to a stable release.  It requires no more skill than clicking on the update button in the radio chooser.  If you want to go backward in revision, it will say do you want to "Downgrade."

There is no requirement to uninstall older versions of the software.  We keep all of the earlier public releases and some of the develop/alpha releases on our work computers.  I have several different versions on my home PC.

I do think you underestimating the complexity of this system when you spoke of a lighter setup module.  There are seven different processors in the SmartSDR system that all need to work in concert and communicate with one another.
Photo of IW7DMH, Enzo

IW7DMH, Enzo

  • 353 Posts
  • 84 Reply Likes
Gerald,

it is difficult for me thinking that all the bugs found in the 1.3.8 version, and all improvements you planned for 1.4 will be magically ready and stable in one weeks or more (just for example). Other bugs will come out but no problems for this.
Probably you could separate release builds from manteinance builds giving to the latter shorter periods. This of course depends on the number of people involved in the project and on the level of coupling between the software levels.
Also, as it is very simple going back and forth between versions, why non give public access at least to beta releases. You already answered on this point, but for me, and many others like me, it is absolutely normal having piece of software not perfectly stable. It happens also to more blazoned systems.
I would also suggest you to keep a public (and possibly numbered) list of known bugs with their workaround or planned version solutions. This surely isn't a great effort and could help people to evaluate your hard work and your progress.

Edit: what is the number of alfa and beta testers involved in SSDR project?
(Edited)
Photo of Gerald - K5SDR

Gerald - K5SDR, Employee

  • 794 Posts
  • 1436 Reply Likes
Enzo, 

Virtually all of the major new functionality was completed in v1.4 by mid December.  We have been focused on system integration, performance tuning and bug fixes since then.

Steve Hicks is the right person instead of me to comment on the development process.  He is too busy with his team to even read the community right now.  

Keeping a public numbered list of known bugs would actually add workload to our process.  We do this to a limited extent on our alpha releases.  We manage our entire software process in Github.  We use terminology and shorthand that would have to be translated for a less technical audience and managed on an almost real time basis.  We don't have spare resources to do that right now.  
Photo of Jay -- N0FB

Jay -- N0FB, Elmer

  • 534 Posts
  • 210 Reply Likes
As much as I love reading Steve's posts filled with technical candy, I'm happy to hear that he has his priorities well established.

As to adding to the Alpha/Beta testing groups:  While having more folks providing bug Information to your development team is great (heck, I'd love to be on the Alpha team), there is an implicit overhead of support required to handle the questions and misunderstandings that any pre-release software causes. (As you have no doubt noticed, we tend to be a bit needy. :-) ) Where the point of diminishing return is for an organization your size, I haven't a clue.  That's where a businessman like yourself needs to rely on instinct, experience, and good business acumen.  

Don't be in a rush to add to the external alpha testing team unless you know it will pay dividends.
(Edited)
Photo of Stan - VA7NF

Stan - VA7NF

  • 411 Posts
  • 90 Reply Likes

Been quiet on this subject until now; some observations:

There will always be complainers.  It changes with the weather between "It takes too long between updates", "You promised ...", "Updates are buggy", "When is function ... coming". 

(From my days with an IBM multi-national assignment on "problem rediscovery reduction") It is much better to get fixes in the field to reduce support overhead (the rediscovery problem) and especially to have solid tested code as a development base.  We had one set of fixes that customers didn't install for two years and in the interim had several new products that were developed on a faulty base.  An extreme example.

Two options may be:

1) A restricted Beta with your week-end development version made available to the Beta Subscription List group for their optional install.  This Beta program would start after the Alpha level completes until public release of the product.  It would be identical to the Alpha process with a different distribution list.

2) An early version (e.g. 1.4.0) made public with the statement that a maintenance release (e.g. 1.4.1) will follow after an early deployment window.  A statement near the top of the announcement notice stating this is an early deployment and some users may wish to delay deployment until the .1 version is available. 

Option 1 carries more support deployment overhead but my experience in problem rediscovery shows that duplicate problem reports drop significantly when timely fixes are made available.  With higher deployment numbers actual field support hours are reduced.

That said, I would participate in an optional Beta program that runs between Alpha end and public (or maintenance .1) release.

Stan Williams - VA7NF

Photo of IW7DMH, Enzo

IW7DMH, Enzo

  • 353 Posts
  • 84 Reply Likes
Please consider also my subscription to an optional Beta program.

Regards
Enzo - iw7dmh
Photo of Bill -VA3WTB

Bill -VA3WTB

  • 2512 Posts
  • 564 Reply Likes
My observations, It is clear the Flex 6000's are very complex machines, more than most on this forum seem to understand. Like wise the SSDR software is very complex. It surprises me how many people post here that has amazing software backgrounds and still can not get their heads around what Gerald and the team is working on. I think most are simply underestimating the complexity of all this.

It also seems to me that Flex has embarked on a steep learning curve and new things pop up they need to work out, they are inventing here!

So for us, we would do well to watch and listen, and enjoy the fruits of there hard work as they come along.
Photo of Peter K1PGV

Peter K1PGV, Elmer

  • 541 Posts
  • 315 Reply Likes
I write software, and manage software engineering projects, for a living.  I've been using Flex gear since the SDR-1000.

I have to say: Given the customer base, the product, and the size of the s/w staff... I think Gerald's team has got things just about right:
  • The have a small, private, test group that "beta tests" the releases and provides them feedback that they would not get from the devs themselves.
  • They hold their releases until what they say will work actually works.
  • They take a step-wise approach to implementing functionality: They obviously strive to implement a broad range of features, and refuse to let perfecting one feature exhaust all available engineering time to the exclusion of doing other features.  Engineers hate this, but it's the right thing for the product.  For example, Flex could have a dev spend months getting the ATU to work to 100% of its potential... which would mean we'd have a working ATU, but no implementation at all of a bunch of other features.
Flex could easily let people download "stable" intermediate builds.  Consider the model that Google uses for releasing the Chrome browser:  You can have the latest release build, the latest stable build, or the latest build.  Pick your poison!

Of course, for Flex to do this, they'd have to add two or three support engineers to deal with the questions, comments, issues, problems, and complaints.  Most of the time would be spent telling folks "Yes sir, you're using the LATEST build... to avoid crashing during the CQ WW DX context you probably should be using the latest RELEASE build."  And then take the pile of complaints from folks who THINK they understand, but don't REALLY understand, the difference.

Yup... I think Flex has things about right.
Photo of Gerald - K5SDR

Gerald - K5SDR, Employee

  • 794 Posts
  • 1436 Reply Likes
Peter, thanks for your thoughtful comments.  

Your last paragraph describes exactly what we experienced in PowerSDR development.  For us to do an expanded beta program, it would require community support in a private forum.  We would need to have a terms of use agreement to in order to participate.   
Photo of IW7DMH, Enzo

IW7DMH, Enzo

  • 353 Posts
  • 84 Reply Likes
"Small private beta tester group" is against the logic of a beta version. I could admit this for alfa versions.
Beta testers should not be overskilled technicians or engineers. Beta version should be used to test all disparate user profiles and should be distributed to a wide audience on the Web. Only in this way you could have a real world test of the software.
Photo of Bill -VA3WTB

Bill -VA3WTB

  • 2512 Posts
  • 564 Reply Likes
Good post Peter, most people really in know think Flex is doing better than anyone expected given the complexity of this new platform.
Photo of SteveM

SteveM

  • 237 Posts
  • 39 Reply Likes
Yeesh... Beta software to a wide audience that has only had a once-over sounds like a good idea until that one bug gets out that fries external amps. Sounds risky.
(Edited)