Discussion:
[wtf] vtscsi aka virtio_scsi does not see disk at target 7
(too old to reply)
Andriy Gapon
2020-01-16 15:27:49 UTC
Permalink
I think I see what's going on, but it was a WTF moment for me.

We use virtio_scsi driver on GCE VMs. We typically configure VMs with 6 or less
disks, but recently we started to use VMs with 8+ disks. On some of them, after
a reboot, one disk is missing. Here is an example:

# camcontrol devlist -v
scbus0 on vtscsi0 bus 0:
<Google PersistentDisk 1> at scbus0 target 1 lun 0 (pass0,da0)
<Google PersistentDisk 1> at scbus0 target 2 lun 0 (pass1,da1)
<Google PersistentDisk 1> at scbus0 target 3 lun 0 (pass2,da2)
<Google PersistentDisk 1> at scbus0 target 4 lun 0 (pass3,da3)
<Google PersistentDisk 1> at scbus0 target 5 lun 0 (pass4,da4)
<Google PersistentDisk 1> at scbus0 target 6 lun 0 (pass5,da5)
<Google PersistentDisk 1> at scbus0 target 8 lun 0 (pass6,da6)
<Google PersistentDisk 1> at scbus0 target 9 lun 0 (pass7,da7)

We noticed that it is always target 7 that is missing.

After looking at the code, the problem is kind of obvious.
scsi_scan_bus:
for (i = low_target; i <= max_target; i++) {


cam_status status;


if (i == initiator_id)


continue;

vtscsi_cam_path_inquiry:
cpi->initiator_id = VTSCSI_INITIATOR_ID;

/*
* Specification doesn't say, use traditional SCSI default.
*/
#define VTSCSI_INITIATOR_ID 7

From what I see, on GCE at least, target 7 is a valid target and it is not
reserved for the initiator. At the same time, I see that target 0 is never used
for disks. Also, it seems that virtio configuration gives 253 as the maximum
target ID on those instances.

I would appreciate any suggestions on what to use for initiator_id in this
driver. Thanks!
--
Andriy Gapon
_______________________________________________
freebsd-***@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
To unsubscribe, send any mail to "freebsd-scsi-***@freebsd.org"

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Andriy Gapon
2020-02-06 10:06:25 UTC
Permalink
By the way, quite a long time ago, on Wed Sep 26 00:57:10 UTC 2012 Jim Harris
I'm wondering if the scsi_xpt.c code
should instead (or also) ignore the initiator_id if cpi->transport ==
XPORT_SAS? initiator_id isn't really valid for SAS transport.
I think that that proposal still makes sense and we should go for it.
Any objections?
I think I see what's going on, but it was a WTF moment for me.
We use virtio_scsi driver on GCE VMs. We typically configure VMs with 6 or less
disks, but recently we started to use VMs with 8+ disks. On some of them, after
# camcontrol devlist -v
<Google PersistentDisk 1> at scbus0 target 1 lun 0 (pass0,da0)
<Google PersistentDisk 1> at scbus0 target 2 lun 0 (pass1,da1)
<Google PersistentDisk 1> at scbus0 target 3 lun 0 (pass2,da2)
<Google PersistentDisk 1> at scbus0 target 4 lun 0 (pass3,da3)
<Google PersistentDisk 1> at scbus0 target 5 lun 0 (pass4,da4)
<Google PersistentDisk 1> at scbus0 target 6 lun 0 (pass5,da5)
<Google PersistentDisk 1> at scbus0 target 8 lun 0 (pass6,da6)
<Google PersistentDisk 1> at scbus0 target 9 lun 0 (pass7,da7)
We noticed that it is always target 7 that is missing.
After looking at the code, the problem is kind of obvious.
for (i = low_target; i <= max_target; i++) {
cam_status status;
if (i == initiator_id)
continue;
cpi->initiator_id = VTSCSI_INITIATOR_ID;
/*
* Specification doesn't say, use traditional SCSI default.
*/
#define VTSCSI_INITIATOR_ID 7
From what I see, on GCE at least, target 7 is a valid target and it is not
reserved for the initiator. At the same time, I see that target 0 is never used
for disks. Also, it seems that virtio configuration gives 253 as the maximum
target ID on those instances.
I would appreciate any suggestions on what to use for initiator_id in this
driver. Thanks!
--
Andriy Gapon
_______________________________________________
freebsd-***@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
To unsubscribe, send any mail to "freebsd-scsi-***@freebsd.org"

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Scott Long
2020-02-06 16:49:36 UTC
Permalink
Hi Andre, sorry for the delay in responding. I agree that initiator_id needs to be ignored. This problem has been avoided in other controllers like mps/mpr by setting the intiator_id to a very high value. Setting it to 0 is liable to be problematic, so it could be set to 254, since 253 is the max target according to the spec. So which would you prefer for now, should vtscsi be changed to give a more correct initiator_id value, or should scsi_xpt.c be changed to ignore the value?

Scott
--
Scott Long
Post by Andriy Gapon
By the way, quite a long time ago, on Wed Sep 26 00:57:10 UTC 2012 Jim Harris
I'm wondering if the scsi_xpt.c code
should instead (or also) ignore the initiator_id if cpi->transport ==
XPORT_SAS? initiator_id isn't really valid for SAS transport.
I think that that proposal still makes sense and we should go for it.
Any objections?
I think I see what's going on, but it was a WTF moment for me.
We use virtio_scsi driver on GCE VMs. We typically configure VMs with 6 or less
disks, but recently we started to use VMs with 8+ disks. On some of them, after
# camcontrol devlist -v
<Google PersistentDisk 1> at scbus0 target 1 lun 0 (pass0,da0)
<Google PersistentDisk 1> at scbus0 target 2 lun 0 (pass1,da1)
<Google PersistentDisk 1> at scbus0 target 3 lun 0 (pass2,da2)
<Google PersistentDisk 1> at scbus0 target 4 lun 0 (pass3,da3)
<Google PersistentDisk 1> at scbus0 target 5 lun 0 (pass4,da4)
<Google PersistentDisk 1> at scbus0 target 6 lun 0 (pass5,da5)
<Google PersistentDisk 1> at scbus0 target 8 lun 0 (pass6,da6)
<Google PersistentDisk 1> at scbus0 target 9 lun 0 (pass7,da7)
We noticed that it is always target 7 that is missing.
After looking at the code, the problem is kind of obvious.
for (i = low_target; i <= max_target; i++) {
cam_status status;
if (i == initiator_id)
continue;
cpi->initiator_id = VTSCSI_INITIATOR_ID;
/*
* Specification doesn't say, use traditional SCSI default.
*/
#define VTSCSI_INITIATOR_ID 7
From what I see, on GCE at least, target 7 is a valid target and it is not
reserved for the initiator. At the same time, I see that target 0 is never used
for disks. Also, it seems that virtio configuration gives 253 as the maximum
target ID on those instances.
I would appreciate any suggestions on what to use for initiator_id in this
driver. Thanks!
--
Andriy Gapon
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
_______________________________________________
freebsd-***@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
To unsubscribe, send any mail to "freebsd-scsi-***@freebsd.org"

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Andriy Gapon
2020-02-07 11:14:24 UTC
Permalink
Post by Scott Long
Hi Andre, sorry for the delay in responding. I agree that initiator_id needs to
be ignored. This problem has been avoided in other controllers like mps/mpr by
setting the intiator_id to a very high value. Setting it to 0 is liable to be
problematic, so it could be set to 254, since 253 is the max target according to
the spec. So which would you prefer for now, should vtscsi be changed to give a
more correct initiator_id value, or should scsi_xpt.c be changed to ignore the
value?
We have already fixed vtscsi.
But I thought that it would be a good idea to do the scsi_xpt change too.
Thanks!
--
Andriy Gapon
_______________________________________________
freebsd-***@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
To unsubscribe, send any mail to "freebsd-scsi-***@freebsd.org"

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Loading...