Possible faulty Azure Sphere device

Thanks for all the suggestions.

I have been through the documentation and the troubleshooting guide suggested by Brian and the guide suggested by Fred but all is setup as suggested.

The issue I have is that the device is now out of the box.

I have setup my AAD properly and also created an account to use with the Global Admin role applied.

When I issue the azsphere login command with that account I get the following which is expected as I haven't created the tenant yet but proves the account is working:

warn: No Azure Sphere tenants found under the selected AAD user.

Successfully logged in with the selected AAD user. This authentication will be used for subsequent commands.

Command completed successfully in 00:00:10.6725886.

Next I do the logical thing and try to create my tenant but this is what is returned:

error: Failed to create an Azure Sphere tenant with name 'AzureSphere'.

Trace ID: acc4be19-6a9f-4133-9fde-073e92f6100b

error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support.

error: Command failed in 00:00:19.5623966.

I have tried using names with spaces, without spaces, with quotes and without and tried random names in case I have hit some reserved term.

I have also tried using the force option and added verbose to see if the logging tells me anything.

Nothing results in a different outcome and the logging just gives me a very long winded "400 Bad Request" result.

I can run the command "azsphere dev show-attached" and get a result describing the device ID and its IP so the comms to the device seems good.

If I log into a different Azure account that I know does already have a tenant created and run the following command I have seen Microsoft suggesting tells us if the device has been pre claimed all seems good.

azsphere dev show-ota-status

Your device is running Azure Sphere OS version 19.07.

error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support.

error: Command failed in 00:00:03.1368024.

Based on Microsofts suggestions it seems this confirms its not claimed.

So what next?

You can only claim a device once and I haven't claimed it anywhere and it seems the above confirms that.

You can only use the device to create one tenant, I haven't created a tenant yet. The account where I had access to a tenant was seeing a tenant created by someone else.

The account with access to a tenant is in a completely different AAD than the one where I'm unable to create a tenant, so different AADs and different accounts.

The reason I suspected hardware is due to MS saying that you can only create a tenant with a device attached and you can only use a device to create a single tenant.

That to me means that something in the devices identity is used to create that tenant and if the device isn't responding properly then that's going to cause a Bad Request response, all supposition but the best I have right now.

I have already run an "azsphere dev recover" to ensure the OS version on the device is up to date and that seems to succeed.

I have checked my AAD config and can see that the 2 Azure Sphere Applications (API and Utility) are listed.

In short I have now run out of places to look and wonder if anyone else can suggest anything or if we think it is a hardware issue does anyone know how Azure Sphere returns work?

Thanks for the detailed response.

Firstly, while logged into each id you used or created, issue:

• azsphere tenant list
• azsphere tenant selected

Then drop in the results here. What we need is the Guid's and names associated with them

Thanks,

Clem

BTW, I worked with multiple devices and learned the hard way how to deal with issues.

For the account with a tenant I get the following:

azsphere tenant list

ID                                   Name

--                                   ----

821b1fe3-aaf7-41bb-9816-ef5b7430124f Emily Sphere 1

azsphere tenant show-selected
Default Azure Sphere tenant ID is '821b1fe3-aaf7-41bb-9816-ef5b7430124f'.

For the account without a tenant I get the following:

azsphere tenant list
warn: No Azure Sphere tenants found under the selected AAD user.
warn: Use 'azsphere login' to change AAD user, or create a new Azure Sphere tenant using 'azsphere tenant create'.
error: You do not have access to any Azure Sphere tenants.
error: Command failed in 00:00:01.3978586.

azsphere tenant show-selected
No default Azure Sphere tenant is selected. If only one Azure Sphere tenant is available
using your current AAD login, it will automatically be used. Otherwise, use this command
to select a default Azure Sphere tenant: 'azsphere tenant select --tenantid <tenant ID>'
Command completed successfully in 00:00:00.3617572.

I would love to get my hands on a few more devices and work through how this all hangs together but when you get a failure to launch on your first device it doesn't inspire you to invest further.

Hopefully we can get over this hurdle and I will procure more devices.

OK, login to the one with the tenant and device attached.

Run this:

azsphere get-support-data --output logs.zip

The log will have debugging information which can show what is true. Post it here and I can interpret it.

The azuresphere_securityservice_213 txt file in zip is the one to look for...

Clem,  really appreciate your help with this.

Command executed and zip file generated.

There was file called "AzureSphere_SecurityService_133.log" and the contents are below:

AZURE SPHERE SECURITY SERVICE CONFIGURATION:
-> Using tenant ID: '821b1fe3-aaf7-41bb-9816-ef5b7430124f'.
-> Using device ID: '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'.

AVAILABLE TENANTS:
-> [821b1fe3-aaf7-41bb-9816-ef5b7430124f] 'Emily Sphere 1'

OVER-THE-AIR CONFIGURATION:
-> Could not get over-the-air configuration: Device with ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3' could not be found.

OVER-THE-AIR UPDATE STATUS:
-> Device OS version(s): 19.07

Could not gather details from the Azure Sphere Security Service: Getting images for device failed.
   at Microsoft.Azure.Sphere.PublicApi.Clients.PublicApiClient.<ListImagesToBeDeployedToDeviceAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Sphere.Tools.SecurityServiceOperations.DeviceManager.<GetSoftwareAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Sphere.Tools.Common.OsVersionChecker.<GetCloudImagesAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Sphere.Tools.Common.OsVersionChecker.<GetTargetedOsVersionAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.Azure.Sphere.Tools.AzSphere.GetSupportDataCommand.<LogOtaStatusAsync>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.Azure.Sphere.Tools.AzSphere.GetSupportDataCommand.<GatherCloudDataAsync>d__25.MoveNext()

Clem,  really appreciate your help with this.

Command executed and zip file generated.

There was file called "AzureSphere_SecurityService_133.log" and the contents are below:

AZURE SPHERE SECURITY SERVICE CONFIGURATION:
-> Using tenant ID: '821b1fe3-aaf7-41bb-9816-ef5b7430124f'.
-> Using device ID: '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'.

AVAILABLE TENANTS:
-> [821b1fe3-aaf7-41bb-9816-ef5b7430124f] 'Emily Sphere 1'

OVER-THE-AIR CONFIGURATION:
-> Could not get over-the-air configuration: Device with ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3' could not be found.

OVER-THE-AIR UPDATE STATUS:
-> Device OS version(s): 19.07

Could not gather details from the Azure Sphere Security Service: Getting images for device failed.
   at Microsoft.Azure.Sphere.PublicApi.Clients.PublicApiClient.<ListImagesToBeDeployedToDeviceAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Sphere.Tools.SecurityServiceOperations.DeviceManager.<GetSoftwareAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Sphere.Tools.Common.OsVersionChecker.<GetCloudImagesAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Sphere.Tools.Common.OsVersionChecker.<GetTargetedOsVersionAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.Azure.Sphere.Tools.AzSphere.GetSupportDataCommand.<LogOtaStatusAsync>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.Azure.Sphere.Tools.AzSphere.GetSupportDataCommand.<GatherCloudDataAsync>d__25.MoveNext()

Here is what I see from above:

1. Your device is not ready for side loading applications. Therefore try this:
   azsphere device prep-debug
2. Your device appears to be claimed by tenant 'Emily Sphere 1'. This is good as far as I can tell.
3. If #1 works, try to load the blinky or hello world scripts to verify functioning.

Let me know how things turn out.

Clem

Thanks for the update but I haven't run a claim as I don't want to use the device there as that's work and the device is for personal testing.

I have specifically avoided claiming until I have this AAD setup with a tenant and will then claim the device into there.

Is this suggesting the device has been associated irrevocably with that tenant just by logging into that account?

To test this further I have run the suggested command to move the device into dev mode and that fails as below suggesting I need to claim the device first.

This would appear to support the fact the device hasn't been claimed and certainly not into the 'Emily Sphere 1' tenant.

I am at risk of sounding like I'm arguing with you which I'm not as I appreciate your knowledge and help but what I know I have done and the evidence I seem to revealing suggests this device isn't claimed but yet wont create the tenant in my AAD into which I will claim it.

More useful logging from these devices when you use the verbose option is going to be essential before I think about proposing this device for real world solutions.

I don't fancy trying to unpick this type pf problem with an SLA clock ticking and the current level of information to work with.

azsphere login
The selected Azure Sphere tenant 'Emily Sphere 1' (821b1fe3-aaf7-41bb-9816-ef5b7430124f) will be retained.
Successfully logged in with the selected AAD user. This authentication will be used for subsequent commands.
Command completed successfully in 00:00:05.5350969.

azsphere device prep-debug
Getting device capability configuration for application development.
Downloading device capability configuration for device ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'.
error: Failed to create device capability configuration for device ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'. Ensure that you are logged in to the correct Azure Sphere tenant ('azsphere login'), that the device is claimed ('azsphere device claim') to that tenant, and try again.
Trace ID: c3542e70-676e-42e6-ac8d-a1fcb9264937
error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support.
error: Command failed in 00:00:02.8190509.

OK, Seems to confirm the claim is not done. So try to create a tenant with  the board plugged in on the correct AAD. If that works, you can claim the device.

As an aside, here is some suggested debugging https://docs.microsoft.com/en-us/azure-sphere/install/troubleshoot-installation .

Please read carefully first: If the claim fails again, carefully take the device id from the zip file we did, and use it without the device attached using the additional parameter. If you make a mistake, it will record that as a device and I see no way of correcting it. So you end up with phantom device!

Clem

I know that doesn't work as that's what kicked this thread off, when I found that action was failing as below:

azsphere tenant create --name "AzureSphere Testing" --force -v
Azure Sphere Utility version 19.5.4.53731
Copyright (C) Microsoft Corporation. All rights reserved.

Start time (UTC): Wednesday, 11 September 2019 14:18:49
verbose: Creating new Azure Sphere tenant with name 'AzureSphere Testing'.
verbose: No Azure Sphere tenants found under the selected AAD user.
verbose: Successfully logged in with the selected AAD user. This authentication will be used for subsequent commands.
verbose: No SerialSlipToTunService port is set in the registry; defaulting to 56665.
verbose: Using attached device ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'.
error: Failed to create an Azure Sphere tenant with name 'AzureSphere Testing'.
Trace ID: 4837d2d7-2e04-4de2-a0bf-3fa46f5f0ec2
Received HTTP response: 400 (Bad Request)
-> Endpoint: POST https://prod.core.sphere.azure.net/tenants
-> Content Type: NULL
error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support. Trace ID: 4837d2d7-2e04-4de2-a0bf-3fa46f5f0ec2

This is exactly as this post https://social.msdn.microsoft.com/Forums/en-US/caea57b7-9f9d-45c6-841b-8a2e718393aa/changing-directory-and-tenant-before…

I suggest you follow this thread for the answer since Microsoft has to fix it.

I am sorry we could not get the ball rolling,

Clem

Unfortunately that post doesn't reflect what's going on here as it seems that device had been used for a one way action previously which mine hasnt.

I had reviewed that one before posting here and was hopeful when I started reading it.

However I have contacted the Microsoft Azure Sphere email address listed there for assistance.

I will update this thread if I ever get a clear answer in case its helpful to others in a similar situation.

Peter,

Can you confirm that you're able to communicate with the Azure Sphere device and read the device ID?  I think I remember seeing a post from you that showed you could.

azsphere device show-attached

I don't think that the device needs to be claimed to execute this command.  If you are able to read the device ID from the device, then the tenant creation issue you're having is likely on Microsoft's side. 

I think you suspect that the device has been previously claimed.  This is highly unlikely.  We manufactured the 20,000 free kits all together and as far as I know, there were not any devices removed from the production run.  All these devices were manufactured, tested and shipped directly to distribution centers.

Brian

Brian,

Yes I can confirm that I get a response from the device that contains a device ID, IP address and connection path.

Peter

Thanks Peter,

That's what I thought.  I've never been through the create tenant process myself, so I gave it a try this morning using a personal email address.  I created a new Azure account using this link in the documentation.  That took me to the interface that signed me up for a new Azure account, created the Azure Active Directory in my account and created my tenant user. 

I can login to the new account using the >"azsphere login" command and the new credentials.  But when I go to create the tenant I get the same error you're getting.  I don't have a unclaimed device with me, so I was using a previously claimed device.  I have a fresh unclaimed device at home and I'll try again on Monday.  I'll post the results here on Monday.  I suspect I'll run into the same issue as you, but I need to try it with a fresh device.

Brian

Thanks Brian, look forward to hearing how things go on Monday.