If mksfs is used under OS/161, the first form should be used, where raw-device is a raw device name (such as "lhd1raw:"). Don't use a device that's already mounted (or being used for swap).
mksfs can also be used on the System/161 host OS, in which case the second form should be used. The host-compiled version of mksfs knows how to deal with the header on System/161 disk images and does the right thing.
Note that as of this writing host-mksfs cannot create disk image files. This is a bug and will hopefully be addressed eventually.