1.4 Suggestion

Jon_KF2E

Since we now know that 1.4 needs to bake a little longer, I was thinking about ways to smooth the process. This version is probably the largest undertaking since SmartSDR itself and was never likely to fit into the schedule for point releases. As a result, I was wondering if it wouldn't be prudent to change how it is released.

Flex has a capable group of alpha testers and I'm sure they represent the most technically competent of current users. Unfortunately, they don't always see things from the perspective of average users. Given the complexity of 1.4, maybe it would be better to give testing another phase. Once 1.4 is working reliably with the alpha testers, release it to a beta with the general population. This would allow users who don't have to have the latest and greatest or are less "cutting edge" the comfort to sit it out and wait for the final release. This would hopefully prevent a surge of support issues and generally make the release of this major update easier to manage. It might also make it easier for management to get it out the door by removing the pressure to chase down every single issue or optimize every line of code.

I know I would sit on the side for a while to see how things are going. I would love to be reading about beta users experiences with 1.4 instead of all the speculation and frustration that is filling the forum of late. Once things were settled with 1.4 I would probably upgrade and get ready for what I'm really waiting for...1.5.

Jon...kf2e

Brent Parker

Jon - That actually not a bad idea, IMHO.

Many of the video driver (Nvidia) sites have posted their current "stable" release, but if you want to try the bleading edge beta, you can. They don't support it in terms of giving response, but certaintly take input.

I've upgraded and downgraded smartsdr several times. It takes a few minutes and is pretty simple, but you need to know a little about what you are doing. For those not used to the un-install process, they might not want to be out there on the bleading edge.

Example, on my work computers, where production is important, I only have "stable" releases. Ocassionally at home if I feel like expermenting, I give beta a try.

So as a flex user, if he just want to operate, I might be better with the stable release. If you want to experiment (in the true ham fashion) then I might jump out there!

There is certaintly a distinction between Alpha and Beta. If you are a Alpha tester, flex is depending on you to test and report back promptly. If you are a Beta user, there is less dependency on you by the company, but as a "enthusiast" you will report discovered anomilities. However as a beta release the company is not obligated to waste time "supporting" issues.

Brent  NB4AP

 

Gerald-K5SDR

Jon and Brent,

Thanks for your thoughts.  You are correct that this release is one of the largest undertakings since the beginning of SmartSDR.  LAN remote is the basis for WAN remote and requires very sophisticated system level integration to make it work right for multiple receivers/panadapters/waterfalls.  All this happens in a real time, client/server, multi-threaded, five processor signal processing system over an unknown network.

In many ways I like the idea of beta releases but I also have some bad experience form the earlier days with PowerSDR.  We did exactly what you were suggesting on PowerSDR until we got burned so many times by customers publicly grumbling about problems with beta software.  It created the perception with potential customers that all our software was in beta, which was not true.  We eventually quit releasing beta on PowerSDR and went to the process we have now for that very reason.  

Frankly, I would love to do as you suggest but I don't know if the general ham population is sophisticated enough to understand the software development process.  I would love to be proven wrong.

73,
Gerald

Joe WD5Y

Gerald, When the 6000 series first came out and it was noted as beta because of the new smartsdr release, I thought it went very well. Some customers waited and many jumped on board myself included in the beta format. To be honest that is what attracted me so much to the powersdr series way back but that is my choice and personal need. I like the idea of a choice for beta or if a person wants to wait. The good thing about this choice is that it puts the ball back into the customer's hands and should lower the grumbling factor a bit. As long as the beta customer agrees and signs into the program with the proper stipulations so everything is clear. I believe beta testing helps a lot because it brings many more operating variables to the table, even with extensive alpha testing done you will still have some bugs because of all of these SDR variables. Thanks so much for all of your time and effort. Sincerely, Joe WD5Y

G4YDO

frankly i'm very tired of all this, can flex answer just one question please, when will fm be released??

Tim - W4TME

FM will be released with SmartSDR v1.4

Jay Nation

Big secret folks, It's all beta, beta all the way down. it will be beta right up until it's final. It will be final just before it becomes historical. Then we move on to SmartSDR v2.0 beta. We still haven't seen PowerSDR v2.7.3 beta. So we have to wait for that too. Your born, you wait, you die. Enjoy the waiting it's all we've got, It's what we do. Besides whining. Whining won't reduce the wait. It entertains some, annoys many. When you stop whining we know your wait is over. Get all the wait you can. 

IW7DMH, Enzo

Dear Mr. Gerald,

as a software developer I agree with you, but as a user of SSDR I think that the release period between two versions is a bit long.
I think also, that, after the 1.4 SSDR version, it could be helpful reviewing the process you release new software.
I would like you stop using the traditional setup process, that often require uninstalling the previous version, and that require users to become very skilled os users. (we all know Windows uninstaller is not a perfect machine).
I would like you adopt a lighter setup module, to be installed only once, and being able to deal itself whit SSDR updates. Just as it already happens with the rig firmware.
In this way developers could release, even daily, small software updates eliminating long waiting periods and requiring little skill and effort from users.
As you use to say these are "just my two cents"
Anyway I like too much my new Flex and I thank you very much for giving us constant updates about your products.

73' Enzo
iw7dmh

G4YDO

yes we know that, 4 months late can we have a idea of the release date mext week, next month, next year, next century, never. we have all paid for this, surely you should keep us more informed.

Ken - NM9P

Hi Enzo,
While this might be desirable, it might not be possible, since SSDR incorporates not only what is in the windows program, but it also installs a new firmware image to the 6000 rig itself.  It might not be possible to do "rolling updates" in something this complex.

Ken - NM9P

Ken - NM9P

Gerald,
I think that there are probably a good many of us who would love to be part of a "second tier" beta testing group. (maybe even higher on the chain)

 I myself would even be willing to sign a "no public kvetching about the beta software" agreement.  (But I reserve the right to publicly address issues in current releases.)  

If someone starts grumbling too much about the beta releases, or requiring too much "hand-holding," they can be dripped from the "team."

I would get a kick out of playing with new releases, even if they are buggy, as long as I can easily revert to something stable when I need it for a contest.  This is one reason I got into Flex SDR's in the first place... to be on the bleeding edge!  Besides, it would be a real hoot to be able to say  "My feedback is why we now have this new function"  or "I helped the team fine tune this element of SSDR."

The question, I gather from your post is, would it take more time to hold our hands than the feedback of extra testers would be worth?

Perhaps there could even be a "closed" or "non-public" section on the community for the beta testing group to help each other and offer feedback to the developers...with the proviso that this is outside the regular "Customer service" loop and that in many ways we would be "on our own."

Thoughts?

Jay Nation

Really? The line is forming directly behind me, now get in line like the rest of us. See you didn't know about the secret line did you Ken. It's because we keep some secrets from you. Why keep secrets? Its how we get people to stand in line, thats why.

Ken - NM9P

Lines?  We don't need no stinkin' lines!  (;P)

Gerald-K5SDR

G4YDO, I would love to give you a date but if my estimate is wrong I am labeled a bad guy.  I personally won't know the exact date until we get through a complete test/debug cycle where there are no remaining critical bugs.  The most I will tell you at this point is that it is weeks, not months.  The guys are making progress every day banging down the issues.  

Jay Nation

Ken
Could you perhaps map out a procedure so that other special people could earn they're own special status. I realize theres nothing fair about being special. There is another, special line. Theres is nobody in it so its a lot shorter. If you go stand in it we can all observe what it's like to be special. Thanks for volunteering. We can't hold your place in this line though. You can stand with us, or go be special. Your choice.  

Gerald-K5SDR

Ken,

You said, 

Perhaps there could even be a "closed" or "non-public" section on the community for the beta testing group to help each other and offer feedback to the developers...with the proviso that this is outside the regular "Customer service" loop and that in many ways we would be "on our own."

This is probably the only way I would be open to setting up a beta testing group.  It would be counter productive for beta to add to our customer service workload or to confuse potential customers with beta related issues.  I do like the idea of gaining a broader test base than we have with the external alpha team.  We would probably want some form of written agreement to join.

None of this is a commitment - just thinking out loud in public.

Gerald-K5SDR

Enzo,

What kind of release cycle are you suggesting?  There is a lot of work that goes into a public release.  Shorter cycles means that the percentage of overhead goes up.  Even doing a weekly alpha release cycle adds engineering overhead.  

By the way, we do real time builds.  Many times from home I grab the latest develop release at home over the VPN and run it.  I usually grab the last one of the day, which includes all of the commits.  I keep all of the older versions installed so that I can go back to a stable release.  It requires no more skill than clicking on the update button in the radio chooser.  If you want to go backward in revision, it will say do you want to "Downgrade."

There is no requirement to uninstall older versions of the software.  We keep all of the earlier public releases and some of the develop/alpha releases on our work computers.  I have several different versions on my home PC.

I do think you underestimating the complexity of this system when you spoke of a lighter setup module.  There are seven different processors in the SmartSDR system that all need to work in concert and communicate with one another.

Jon_KF2E

Gerald,

If you do decide to do this I would gladly sign up. Recently retired, I spend a large portion of my time with my radio.

Jon...kf2e

Ken - NM9P

Gerald, I understand about the "thinking out loud" part.  
I have gotten stung a few time in board meetings while doing that!

If you get to the point of actually implementing it, please sign me up!

It think it could go along with the spirit in the video of your presentation that Howard recently posted . . .about the two different markets in the SDR field - the bleeding edge experimenters, and the high performance operators.  I find I am putting down roots on both sides of the "chasm."  The problem comes when people forget which level of support they should expect from each side of the chasm.   

One of my biggest kicks in SDR was being asked to help test some of Steve's renditions of APF and other CW functions along with NN4ZZ and others in the pre-release and early post-1.0 release  days.  I don't know how helpful I actually was, but it was fun.  I have much more time in the seat on the 6500 now and might be able to be add more to the evaluation now!

Again... I certainly wouldn't want any of this to hinder current development efforts.  That wouldn't be "SmartSDR" policy!

IW7DMH, Enzo

Gerald,

it is difficult for me thinking that all the bugs found in the 1.3.8 version, and all improvements you planned for 1.4 will be magically ready and stable in one weeks or more (just for example). Other bugs will come out but no problems for this.
Probably you could separate release builds from manteinance builds giving to the latter shorter periods. This of course depends on the number of people involved in the project and on the level of coupling between the software levels.
Also, as it is very simple going back and forth between versions, why non give public access at least to beta releases. You already answered on this point, but for me, and many others like me, it is absolutely normal having piece of software not perfectly stable. It happens also to more blazoned systems.
I would also suggest you to keep a public (and possibly numbered) list of known bugs with their workaround or planned version solutions. This surely isn't a great effort and could help people to evaluate your hard work and your progress.

Edit: what is the number of alfa and beta testers involved in SSDR project?

Gerald-K5SDR

Enzo, 

Virtually all of the major new functionality was completed in v1.4 by mid December.  We have been focused on system integration, performance tuning and bug fixes since then.

Steve Hicks is the right person instead of me to comment on the development process.  He is too busy with his team to even read the community right now.  

Keeping a public numbered list of known bugs would actually add workload to our process.  We do this to a limited extent on our alpha releases.  We manage our entire software process in Github.  We use terminology and shorthand that would have to be translated for a less technical audience and managed on an almost real time basis.  We don't have spare resources to do that right now.  

Stan VA7NF

Been quiet on this subject until now; some observations:

There will always be complainers.  It changes with the weather between "It takes too long between updates", "You promised ...", "Updates are buggy", "When is function ... coming". 

(From my days with an IBM multi-national assignment on "problem rediscovery reduction") It is much better to get fixes in the field to reduce support overhead (the rediscovery problem) and especially to have solid tested code as a development base.  We had one set of fixes that customers didn't install for two years and in the interim had several new products that were developed on a faulty base.  An extreme example.

Two options may be:

1) A restricted Beta with your week-end development version made available to the Beta Subscription List group for their optional install.  This Beta program would start after the Alpha level completes until public release of the product.  It would be identical to the Alpha process with a different distribution list.

2) An early version (e.g. 1.4.0) made public with the statement that a maintenance release (e.g. 1.4.1) will follow after an early deployment window.  A statement near the top of the announcement notice stating this is an early deployment and some users may wish to delay deployment until the .1 version is available. 

Option 1 carries more support deployment overhead but my experience in problem rediscovery shows that duplicate problem reports drop significantly when timely fixes are made available.  With higher deployment numbers actual field support hours are reduced.

That said, I would participate in an optional Beta program that runs between Alpha end and public (or maintenance .1) release.

Stan Williams - VA7NF

IW7DMH, Enzo

Please consider also my subscription to an optional Beta program.

Regards
Enzo - iw7dmh

Jay -- N0FB

As much as I love reading Steve's posts filled with technical candy, I'm happy to hear that he has his priorities well established.

As to adding to the Alpha/Beta testing groups:  While having more folks providing bug Information to your development team is great (heck, I'd love to be on the Alpha team), there is an implicit overhead of support required to handle the questions and misunderstandings that any pre-release software causes. (As you have no doubt noticed, we tend to be a bit needy. :-) ) Where the point of diminishing return is for an organization your size, I haven't a clue.  That's where a businessman like yourself needs to rely on instinct, experience, and good business acumen.  

Don't be in a rush to add to the external alpha testing team unless you know it will pay dividends.

Bill -VA3WTB

My observations, It is clear the Flex 6000's are very complex machines, more than most on this forum seem to understand. Like wise the SSDR software is very complex. It surprises me how many people post here that has amazing software backgrounds and still can not get their heads around what Gerald and the team is working on. I think most are simply underestimating the complexity of all this.

It also seems to me that Flex has embarked on a steep learning curve and new things pop up they need to work out, they are inventing here!

So for us, we would do well to watch and listen, and enjoy the fruits of there hard work as they come along.

Peter K1PGV

I write software, and manage software engineering projects, for a living.  I've been using Flex gear since the SDR-1000.

I have to say: Given the customer base, the product, and the size of the s/w staff... I think Gerald's team has got things just about right:

• The have a small, private, test group that "beta tests" the releases and provides them feedback that they would not get from the devs themselves.
• They hold their releases until what they say will work actually works.
• They take a step-wise approach to implementing functionality: They obviously strive to implement a broad range of features, and refuse to let perfecting one feature exhaust all available engineering time to the exclusion of doing other features.  Engineers hate this, but it's the right thing for the product.  For example, Flex could have a dev spend months getting the ATU to work to 100% of its potential... which would mean we'd have a working ATU, but no implementation at all of a bunch of other features.

Flex could easily let people download "stable" intermediate builds.  Consider the model that Google uses for releasing the Chrome browser:  You can have the latest release build, the latest stable build, or the latest build.  Pick your poison!

Of course, for Flex to do this, they'd have to add two or three support engineers to deal with the questions, comments, issues, problems, and complaints.  Most of the time would be spent telling folks "Yes sir, you're using the LATEST build... to avoid crashing during the CQ WW DX context you probably should be using the latest RELEASE build."  And then take the pile of complaints from folks who THINK they understand, but don't REALLY understand, the difference.

Yup... I think Flex has things about right.

IW7DMH, Enzo

"Small private beta tester group" is against the logic of a beta version. I could admit this for alfa versions.
Beta testers should not be overskilled technicians or engineers. Beta version should be used to test all disparate user profiles and should be distributed to a wide audience on the Web. Only in this way you could have a real world test of the software.

Bill -VA3WTB

Good post Peter, most people really in know think Flex is doing better than anyone expected given the complexity of this new platform.

Gerald-K5SDR

Peter, thanks for your thoughtful comments.  

Your last paragraph describes exactly what we experienced in PowerSDR development.  For us to do an expanded beta program, it would require community support in a private forum.  We would need to have a terms of use agreement to in order to participate.   

SteveM

Yeesh... Beta software to a wide audience that has only had a once-over sounds like a good idea until that one bug gets out that fries external amps. Sounds risky.