Spot API timestamp

Mark_WS7M

Hi All,

I've been working on my own software to deal with logging and spots.  I installed 2.3.9 so I have the latest spot fixes.

I'm seeing something strange with the spot timestamp or perhaps I don't know what I am supposed to do with it.

The current GMT as I write this is around 12:45Z.   As I see a spot come in from either SliceMaster or a spot network to my software I can see the timestamp on the data read from the server is correct... Right around 1245Z or so.

So my software constructs a radio Spot.  My understanding of a timestamp would be the current UTC date time which would be 07/10/2018 and the time should be the UTC time of the spot as read from the telnet stream:   1246 as an example.

So I construct my date time object with 07 10 2018 12 46 00 Z.   When I examine this date time object in the debugger it is correct.

I assign it to the spot timestamp field.  I submit the spot to the radio and it shows up.  However, hover over the spot and the spot date/time shows:  07/10/2018 18:46.   So +6 hours.

This almost looks like the spot timestamp in the Spot object is based on local time and the API or radio is trying to convert it to GMT.

Is that what is happening?  If so I guess I am wondering since all spot data just about everywhere is based on UTC time why would the spot object/radio need a local time?

K1DBO

Mark,

The problem is probably at the point where you convert your "date time object" to a Unix timestamp (seconds since the Epoch) for delivery to the Flex or possibly in the construction of the date time object itself.  You dont mention the language you are using but from your description, you probably dont need a date time object at all, just a direct call to a system time() function that returns the Unix timestamp.

Now with that out of the way... I dont think you want the time that the spot was read from the telnet stream.  More likely you want the time that the spot was created.  The creation time is added by the sever that first received the spot and is passed along over the telnet connection to the connected clients.  But depending on the format in which the cluster is sending data, that timestamp might only include and hour:minute field (in GMT).  So around midnight, there is an interesting edge you have to watch out for when converting that to a Unix timestamp.

To answer your question more directly... the timestamp field in the Flex API spot command is expecting a Unix timestamp.  By definition, this is UTC.  

The display of the timestamp in the tool tip happens in SSDR.  I suspect they are converting it to GMT for display.  At the very least, it corresponds to the time display at the lower right corner of my SSDR window.  I forget if SSDR provides an option to change that to local time, but if it does, I'd expect that option to be respected by the spot tool tip.

Wo, oh, what I want to know, where does the time go?

Good luck!

--Don

Mark WS7M

Hi Don,

I am using the time read from the telnet stream... meaning if the telnet stream is:

DX de NC6R:   14218.0   VU2MTM    Listen for West Coast   1508Z

I am grabbing the 1508Z.

I construct a C# datetime object which is DateTime.UtcNow which gives the correct date and time of right now in GMT.

I then modify the hour and minute to be 15 and 08.  I have yet to deal with 0000 rollover but will once I have it basically working.

I then assign that the the DateTime field of the Spot object in the radio C# API.

The spot appears on the radio but the spot date/time has been adjusted by my local GMT offset which currently is 

I've confirmed that this is happening somewhere in the path between the "RequestSpot()" call in the Radio API and the radio itself.

If I send my local datetime then the GMT time shows correctly on the spot.

So my question I think is still valid... Why if spots come in based on GMT is the API (provided by flex) expecting a local DateTime to be assigned to the Spot object?

Thanks for replying but something you said kind of doesn't make sense.  You said the radio API spot is expecting a unix timestamp which by definition is in UTC.  But then later you said they are probably converting the tool tip to UTC. 

That in itself could be an issue unless the radio takes the unix timespace in UTC converts it back to local for storage then converts it again back to UTC in the tool tip.

I am using the C# based API that Flex provides.  The Spot object which you are supposed to populate BEFORE calling the Radio.RequestSpot() method.

Spot has a field called Timestamp for which the summary help shows:

"The timestamp (UTC) of the spot as reported by the original source"

This "Timestamp" field in the C# API object Spot is of type "DateTime".  I can only assign a DateTime object to it.

In C# the DateTime class has a static method called:   UtcNow   This returns a DateTime object populated with the current date and time in UTC.  The values in this object MATCH the displayed UTC time shown on the radio when examined in the debugger.

If I assign DateTime.UtcNow to the Timestamp field of object Spot.  Then RequestSpot() from the radio.  The spot on the pan shows up but the time has been adjusted in my case +6 hours.

So I believe there is an issue in the API provided by flex that even though Timestamp in the Spot object says it wants UTC time, it really wants local time.

I do believe this is a bug in the Flex API not in something I am doing.

To confirm as I mentioned I think, if I assign Timestamp = DateTime.Now which is local date/time then the spot shows up with the CORRECT UTC time.

This is not a huge problem but I think the bug should be fixed.  If the Timestamp says it wants UTC it should actually take UTC.

So what I must do now to work around this is when, as in the spot above I get the spotted time of 1508Z I need to convert this to my local time before I assign it to the spot object.

K1DBO

I havent used the Flex C# API.  I code direct to the radio's TCP port.  I often read the Flex C# API code for reference as it's the best documentation available.  Is the source for the lasted C# API available?  I'll take a look at what's going on there if it is.

Now... apologies for asking without reading the code first... what you are saying implies that the C# API is looking for a datetime object to be passed in for spot creation.  If so, it sounds like what you are doing is correct.  Simply passing in a "UTC now" object should result in the spot tool tip showing the current time in UTC.  If not, there is a bug in the C# API where it's not sending the right thing to the radio. 

It sounds like you've done all that and reached the conclusion that there is a bug in the Flex C# API.  I agree.

Regarding the bit that "doesnt make sense"... remember, you are not talking directly to SSDR, but SSDR is displaying the spot.  Your program is sending a spot to the radio.  To do that the spot's timestamp must be converted to a Unix timestamp.  The radio receives the spot then generates a status message to be sent to all clients subscribed to spot updates.  SSDR is one of these clients.  When SSDR receives the status update for the new spot, it must extract the timestamp, format it, and display it as a tooltip.  The conversion I was referring to was from the Unix timestamp (seconds since the Epoch) to a date time object in C# and finally to a text representation for display.

I know that Flex gets at least half of this right.  Slice Master simply sends a Unix timestamp to the radio.  When it finally makes it back to SSDR for display, it does what I would expect.  While you are taking a different route, it seems your expectations are the same but not being met.  The best explanation I can see for this is that there is a bug in the Flex C# API.

--Don

Eric-KE5DTO

The Timestamp field in the Spot class is a DateTime object.  In C#, the DateTime object doesn't have an associated time zone, but the description for the field shows that the object represents UTC time already.  So doing a .UtcNow would double compensate for the delta between the local time and UTC.

Mark WS7M

Ok I'm confused... So I should assign it local time as I'm doing now and it will take care of it?

Eric-KE5DTO

In short, yes.

Longer answer: I think the DateTime object you are using is already in UTC based on what you have said (it comes from a telnet source as UTC).  You should be able to use that time directly since UTC is what the field already wants.  By taking the DateTime object that is already UTC and doing UtcNow on it, it adds an additional offset for the delta between the local time (which it assumes is what your DateTime object represents) and UTC.  This ends up double confusing things.

Mark_WS7M

Hi Eric,

I don't think that is what is happening and this could all be sort of a misunderstanding of what the spot object wants.

As I mentioned above the spot object when hover over the Timestamp field says UTC and to me, in 40 years of development that means I should assign a UTC date/time to that field.  I don't think I've ever had a case where something has hint help saying UTC but expects to get a non UTC time and will convert it.

So here is the test I ran:

Very simple program that just connects to the API.

I create a Spot object called "spot".  I set my call as the spotted call, set frequency say of 7.056

I assign

    spotSpot.Timestamp = DateTime.UtcNow;

As I am sure you are well aware if you in code do:

    DateTime myDateTime = DateTime.UtcNow;

and you examine myDateTime in the debugger the date and time fields will match the current GMT date/time (assuming your computer is setup reasonable).

In my case the values in myDateTime match the date/time displayed at the lower right of SSDR.   

So this tells me that myDateTime contains the current GMT (or UTC) date/time.

Since Spot.Timestamp hint says UTC if I either assign DateTime.UtcNow to it or take the long approach, assign it to myDateTime then assign that to Spot.Timestamp and then call:

Radio.RequestSpot( spot );

My spot will show up at 7.056 like I expect however the spotted time shown if I hover over the spot will be +7 hours ahead of current GMT.  So if the radio says 02:38Z my spot will show 09:38Z.

If I do the exact same test but this time when assigning TimeStamp I simply do:

spot.Timestamp = DateTime.Now

Where DateTime.Now will match the current local date time of the computer if looked at in the debugger then the spot will show up on the pan and the spot time will correctly be in UTC.

So I think the hint for the Timestamp is wrong.  It should say (Local) time since it seems to be doing the GMT conversion.  Or the API needs to be fixed so it does not convert the UTC time assigned to it.

That's my take on it.   I currently have my code working but I have to take incoming spot times from spot server which are in GMT and I have to convert them to local time before assigning them to spot.Timestamp.

K1DBO

Mark, 

This code,

        private DateTime UnixTimestampToDateTime(long timestamp)
        {
            DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);
            return origin.AddSeconds(timestamp);
        }
        internal long DateTimeToUnixTimestamp(DateTime date)
        {
            DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);
            TimeSpan diff = date.ToUniversalTime() - origin;
            return (long)diff.TotalSeconds;
        }

Is from the 2.3.9 FlexLib C# API.  To my eye, it looks like it's doing the right thing in both directions.

Note that the origin (Epoch) is expressed as a UTC value and that the value being passed in from the spot is being converted to UTC before the delta is computed.  That delta expressed in seconds is a valid Unix timestamp.  This code is from Spot.cs and is called from Radio.cs.  The returned value is simply passed to the radio in a 'spot add' command.

There's a comment in Spot.cs documenting the Timestamp property.  It mentions UTC, but I'm not seeing anywhere that makes that a requirement.  The timestamp could be either local time or UTC and the DateTimeToUnixTimestamp() method should  do the right thing.  

Unless I'm misreading the code, the Flex C# API sends timestamps to the radio exactly as I would expect it too.  Based on your previous comments, it would seem that is also what you would expect.  Now, here's the rub.  It seems that your code is not doing what you expect it to!

So, where's the bug?  If I had a C# environment set up, I'd test that the DateTime object I was assigning to the spot had its Kind property set to DateTimeKind.Utc.  Because when it does, the ToUniversalTime() method should not perform a conversion (per MS docs).  I'd also verify that 

    my_datetime.ToUniversalTime() == my_datetime

when

    my_datetime.Kind == DateTimeKind.Utc

If both those conditions hold and you are still getting the wrong date displayed in SSDR... mark me down as confused.  I'd be really surprised if DateTime.UtcNow returned a DateTime object with Kind set to anything but Utc.  Yet, that would explain what you are seeing.

Eric-KE5DTO

I just did a test where I created a Spot object using DateTime.UtcNow and it showed up correctly in the tooltip for me once I called radio.RequestSpot with the object.

Mark WS7M

Ya I just verified the same.

Any chance this was somehow fixed in 2.3.9?  I think I was on 2.3.7 before

Anyway consider this closed.  Thanks for everyone looking into it.

Eric-KE5DTO

I don't think it has changed.  The timezone stuff definitely will make your head spin though.  I'm glad it's working now.