for CXL device create-namespace showing failed to create namespace: No space left on device

[suyoggatkal20]

I am getting this error while creating namespace for cxl device.
It worked once with same procedure given below.
But now it is not working.
what could be the reasons?
why this is behaving in non-deterministic way.
i have build the kernel with proper configuration needed for cxl and running it on qemu.
My host as well as guest is Ubuntu 22.04 LTS.
i have followed procedure provided here: https://github.com/linuslau/CXL-Emulator-QEMU?tab=readme-ov-file

here is the commands output of few relevant commands.

lspci -vvv -s 0d:00.0
0d:00.0 CXL: Intel Corporation Device 0d93 (rev 01) (prog-if 10 [CXL Memory Device (CXL 2.x)])
Subsystem: Red Hat, Inc. Device 1100
Physical Slot: 2
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Latency: 0
Region 0: Memory at fe800000 (64-bit, non-prefetchable) [size=64K]
Region 2: Memory at fe810000 (64-bit, non-prefetchable) [size=4K]
Region 4: Memory at fe811000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [40] MSI-X: Enable+ Count=6 Masked-
Vector table: BAR=4 offset=00000000
PBA: BAR=4 offset=00000800
Capabilities: [80] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Exit Latency L0s <64ns
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 4
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Designated Vendor-Specific: Vendor=1e98 ID=0000 Rev=3 Len=60: CXL
CXLCap: Cache- IO+ Mem+ Mem HW Init+ HDMCount 1 Viral-
CXLCtl: Cache- IO+ Mem+ Cache SF Cov 0 Cache SF Gran 0 Cache Clean- Viral-
CXLSta: Viral-
Capabilities: [13c v1] Designated Vendor-Specific: Vendor=1e98 ID=0008 Rev=0 Len=36 Capabilities: [160 v1] Designated Vendor-Specific: Vendor=1e98 ID=0005 Rev=0 Len=16
Capabilities: [170 v1] Designated Vendor-Specific: Vendor=1e98 ID=0007 Rev=2 Len=32 <?>
Capabilities: [190 v1] Extended Capability ID 0x2e
Capabilities: [200 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Kernel driver in use: cxl_pci

cxl/cxl list -u
{
"memdev":"mem0",
"pmem_size":"512.00 MiB (536.87 MB)",
"pmem_qos_class":0,
"serial":"0",
"host":"0000:0d:00.0",
"firmware_version":"BWFW VERSION 00"
}

cxl/cxl list -Mu
{
"memdev":"mem0",
"pmem_size":"512.00 MiB (536.87 MB)",
"pmem_qos_class":0,
"serial":"0",
"host":"0000:0d:00.0",
"firmware_version":"BWFW VERSION 00"
}

cxl create-region -m -d decoder0.0 -w 1 mem0 -s 512M
[ 106.298526] cxl_core:cxl_region_probe:3269: cxl_region region0: config state: 02M
[ 106.301070] cxl_core:cxl_bus_probe:2077: cxl_region region0: probe: -6
[ 106.306190] cxl_core:devm_cxl_add_region:2429: cxl_acpi ACPI0017:00: decoder0.0: created region0
[ 106.318897] cxl_core:cxl_dpa_alloc:685: cxl decoder2.0: DPA Allocation start: 0x0000000000000000 len: 0x20000000 Ski0
[ 106.324565] cxl_core:__cxl_dpa_reserve:450: cxl_port endpoint2: decoder2.0: [mem 0x00000000-0x1fffffff flags 0x800002
[ 106.330817] cxl_core:cxl_port_attach_region:1010: cxl region0: mem0:endpoint2 decoder2.0 add: mem0:decoder2.0 @ 0 ne1
[ 106.337080] cxl_core:cxl_port_attach_region:1010: cxl region0: pci0000:0c:port1 decoder1.0 add: mem0:decoder2.0 @ 0 1
[ 106.342502] cxl_core:cxl_port_setup_targets:1267: cxl region0: pci0000:0c:port1 iw: 1 ig: 256
[ 106.345763] cxl_core:cxl_port_setup_targets:1291: cxl region0: pci0000:0c:port1 target[0] = 0000:0c:00.0 for mem0:de0
[ 106.349963] cxl_core:cxl_calc_interleave_pos:1847: cxl_mem mem0: decoder:decoder2.0 parent:0000:0d:00.0 port:endpoin0
[ 106.354641] cxl_core:cxl_region_attach:2048: cxl decoder2.0: Test cxl_calc_interleave_pos(): success test_pos:0 cxle0
[ 106.358149] cxl region0: Bypassing cpu_cache_invalidate_memregion() for testing!
[ 106.359895] cxl_core:cxl_bus_probe:2077: cxl_pmem_region pmem_region0: probe: 0
[ 106.362876] cxl_core:devm_cxl_add_pmem_region:2946: cxl_region region0: region0: register pmem_region0
[ 106.366012] cxl_core:cxl_bus_probe:2077: cxl_region region0: probe: 0
{
"region":"region0",
"resource":"0xa90000000",
"size":"512.00 MiB (536.87 MB)",
"type":"pmem",
"interleave_ways":1,
"interleave_granularity":256,
"decode_state":"commit",
"mappings":[
{
"position":0,
"memdev":"mem0",
"decoder":"decoder2.0"
}
]
}
cxl region: cmd_create_region: created 1 region

ndctl/ndctl create-namespace -m dax -r region0
failed to create namespace: No space left on device

[sscargal]

@suyoggatkal20 If you are using files as your backing store on the host, and did not remove those files before the second attempt, then everything you did in the first execution will still be there. This is likely why your 2nd attempt failed with ENOSPC: it's already 100% allocated.

From the qemu command you reference, it creates the backing store in /tmp on the host:

    -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=512M \
    -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=512M \

If the host wasn't rebooted between the 1st and 2nd attempt, these files are still there and being referenced/used.

Depending on what you want to do next/now, on the host, you can either delete the backing files and recreate them, restart the QEMU instance, and go through your procedure again. This assumes you don't want/need the data written in the first attempt. Or, within the QEMU guest VM, you can skip the namespace creation if it already exists (ndctl list), or destroy the existing dax namespace and re-create it.

[AlisonSchofield]

@suyoggatkal20 Did Steve's suggestion above resolve your issue?