Flash access on iCEBreaker board?

[agamez]

Hi!

I am implementing a system using the iCEBreaker board. I got neorv32 running and I can upload programs via serial port using the bootloader. All seems fine! Now, I want to store the software on flash, and I'm facing issues while accessing flash chip:

<< NEORV32 Bootloader >>

BLDV: Mar 14 2023
HWV:  0x01080000
CID:  0x00000000
CLK:  0x017d7840
ISA:  0x40800100 + 0x00000081
SOC:  0x100d000d
IMEM: 0x00000400 bytes @0x00000000
DMEM: 0x00000400 bytes @0x80000000Available CMDs:
 h: Help
 r: Restart
 u: Upload
 s: Store to flash
 l: Load from flash
 x: Boot from flash (XIP)
 e: Execute
CMD:> l
Loading (@0x00400000)...
ERR_FLSH

I believe to have properly set constraints:

This is the pinout

#> SPI Flash
set_io --warn-no-port flash_mosi 14
set_io --warn-no-port flash_sck 15
set_io --warn-no-port flash_miso 17
set_io --warn-no-port spi_csn[0] 16

But maybe my VHDL code is wrong somehow, because there's a little quirk about my design: I need SPI access to both the FLASH and also to an external device connected via one of the PMODs, so I had to multiplex some SPI signals out of my design:

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

library iCE40;
use iCE40.components.all; -- for device primitives and macros

library neorv32;
use neorv32.neorv32_package.all;

entity art_gs_top is
	port (
		rstn : in std_logic;

		uart_rx : in std_logic;
		uart_tx : out std_logic;

		spi_miso : in std_logic;
		spi_mosi : out std_logic;
		spi_sck : out std_logic;

		flash_miso : in std_logic;
		flash_mosi : out std_logic;
		flash_sck : out std_logic;

		spi_csn : out std_ulogic_vector(3 downto 0)
	);
end entity;

architecture art_gs_top_rtl of art_gs_top is
	-- configuration --
	constant f_clock_c : natural := 25000000; -- PLL output clock frequency in Hz

	-- On-chip oscillator --
	signal hf_osc_clk : std_logic;

	-- Globals
	signal pll_rstn : std_logic;
	signal pll_clk  : std_logic;

	-- User signals
	signal spi_miso_int : std_logic;
	signal spi_mosi_int : std_logic;
	signal spi_sck_int : std_logic;
	signal spi_csn_int : std_logic_vector(7 downto 0);

begin

	-- On-Chip HF Oscillator ------------------------------------------------------------------
	-- -------------------------------------------------------------------------------------------
	HSOSC_inst : SB_HFOSC
	generic map (
		CLKHF_DIV => "0b00" -- 48 MHz
	)
	port map (
		CLKHFPU => '1',
		CLKHFEN => '1',
		CLKHF   => hf_osc_clk
	);

	-- System PLL -----------------------------------------------------------------------------
	-- -------------------------------------------------------------------------------------------
	-- Settings generated by icepll -i 48 -o 25:
	-- F_PLLIN:      48.000 MHz (given)
	-- F_PLLOUT:     25.000 MHz (requested)
	-- F_PLLOUT:     25.000 MHz (achieved)
	-- FEEDBACK:     SIMPLE
	-- F_PFD:        16.000 MHz
	-- F_VCO:        800.000 MHz
	-- DIVR:         2 (4'b0010)
	-- DIVF:        49 (7'b0110001)
	-- DIVQ:         5 (3'b101)
	-- FILTER_RANGE: 1 (3'b001)
	Pll_inst : SB_PLL40_CORE
	generic map (
		FEEDBACK_PATH => "SIMPLE",
		DIVR          =>  x"2",
		DIVF          => "0110001",
		DIVQ          => 3x"5",
		FILTER_RANGE  => 3x"1"
	)
	port map (
		REFERENCECLK    => hf_osc_clk,
		PLLOUTCORE      => open,
		PLLOUTGLOBAL    => pll_clk,
		EXTFEEDBACK     => '0',
		DYNAMICDELAY    => x"00",
		LOCK            => pll_rstn,
		BYPASS          => '0',
		RESETB          => rstn,
		LATCHINPUTVALUE => '0',
		SDO             => open,
		SDI             => '0',
		SCLK            => '0'
	);

	neorv32_top_inst: entity neorv32.neorv32_top
	generic map (
		-- General --
		CLOCK_FREQUENCY              => f_clock_c,      -- clock frequency of clk_i in Hz
		INT_BOOTLOADER_EN            => true,   -- boot configuration: true = boot explicit bootloader; false = boot from int/ext (I)MEM
		CUSTOM_ID                    => x"00000000", -- custom user-defined ID
		HW_THREAD_ID                 => 0,      -- hardware thread id (32-bit)
		-- On-Chip Debugger (OCD) --
		ON_CHIP_DEBUGGER_EN          => false,  -- implement on-chip debugger
		-- RISC-V CPU Extensions --
		CPU_EXTENSION_RISCV_B        => false,  -- implement bit-manipulation extension?
		CPU_EXTENSION_RISCV_C        => false,  -- implement compressed extension?
		CPU_EXTENSION_RISCV_E        => false,  -- implement embedded RF extension?
		CPU_EXTENSION_RISCV_M        => false,  -- implement muld/div extension?
		CPU_EXTENSION_RISCV_U        => false,  -- implement user mode extension?
		CPU_EXTENSION_RISCV_Zfinx    => false,  -- implement 32-bit floating-point extension (using INT reg!)
		CPU_EXTENSION_RISCV_Zicsr    => true,   -- implement CSR system?
		CPU_EXTENSION_RISCV_Zicntr   => true,   -- implement base counters?
		CPU_EXTENSION_RISCV_Zihpm    => false,  -- implement hardware performance monitors?
		CPU_EXTENSION_RISCV_Zifencei => false,  -- implement instruction stream sync.?
		CPU_EXTENSION_RISCV_Zmmul    => false,  -- implement multiply-only M sub-extension?
		CPU_EXTENSION_RISCV_Zxcfu    => false,  -- implement custom (instr.) functions unit?
		-- Extension Options --
		FAST_MUL_EN                  => false,  -- use DSPs for M extension's multiplier
		FAST_SHIFT_EN                => false,  -- use barrel shifter for shift operations
		CPU_IPB_ENTRIES              => 1,      -- entries in instruction prefetch buffer, has to be a power of 2, min 1
		-- Physical Memory Protection (PMP) --
		PMP_NUM_REGIONS              => 0,      -- number of regions (0..16)
		PMP_MIN_GRANULARITY          => 4,      -- minimal region granularity in bytes, has to be a power of 2, min 4 bytes
		-- Hardware Performance Monitors (HPM) --
		HPM_NUM_CNTS                 => 0,      -- number of implemented HPM counters (0..29)
		HPM_CNT_WIDTH                => 40,     -- total size of HPM counters (0..64)
		-- Internal Instruction memory --
		MEM_INT_IMEM_EN              => true,   -- implement processor-internal instruction memory
		MEM_INT_IMEM_SIZE            => 1*1024, -- size of processor-internal instruction memory in bytes
		-- Internal Data memory --
		MEM_INT_DMEM_EN              => true,   -- implement processor-internal data memory
		MEM_INT_DMEM_SIZE            => 1*1024, -- size of processor-internal data memory in bytes
		-- Internal Cache memory --
		ICACHE_EN                    => false,  -- implement instruction cache
		ICACHE_NUM_BLOCKS            => 4,      -- i-cache: number of blocks (min 1), has to be a power of 2
		ICACHE_BLOCK_SIZE            => 64,     -- i-cache: block size in bytes (min 4), has to be a power of 2
		ICACHE_ASSOCIATIVITY         => 1,      -- i-cache: associativity / number of sets (1=direct_mapped), has to be a power of 2
		-- External memory interface --
		MEM_EXT_EN                   => false,  -- implement external memory bus interface?
		MEM_EXT_TIMEOUT              => 255,    -- cycles after a pending bus access auto-terminates (0 = disabled)
		MEM_EXT_PIPE_MODE            => false,  -- protocol: false=classic/standard wishbone mode, true=pipelined wishbone mode
		MEM_EXT_BIG_ENDIAN           => false,  -- byte order: true=big-endian, false=little-endian
		MEM_EXT_ASYNC_RX             => false,  -- use register buffer for RX data when false
		MEM_EXT_ASYNC_TX             => false,  -- use register buffer for TX data when false
		-- Stream link interface --
		SLINK_NUM_TX                 => 0,      -- number of TX links (0..8)
		SLINK_NUM_RX                 => 0,      -- number of TX links (0..8)
		SLINK_TX_FIFO                => 1,      -- TX fifo depth, has to be a power of two
		SLINK_RX_FIFO                => 1,      -- RX fifo depth, has to be a power of two
		-- External Interrupts Controller (XIRQ) --
		XIRQ_NUM_CH                  => 0,      -- number of external IRQ channels (0..32)
		XIRQ_TRIGGER_TYPE            => (others => '1'), -- trigger type: 0=level, 1=edge
		XIRQ_TRIGGER_POLARITY        => (others => '1'), -- trigger polarity: 0=low-level/falling-edge, 1=high-level/rising-edge
		-- Processor peripherals --
		IO_GPIO_EN                   => true,   -- implement general purpose input/output port unit (GPIO)?
		IO_MTIME_EN                  => false,   -- implement machine system timer (MTIME)?
		IO_UART0_EN                  => true,   -- implement primary universal asynchronous receiver/transmitter (UART0)?
		IO_UART0_RX_FIFO             => 2,      -- RX fifo depth, has to be a power of two, min 1
		IO_UART0_TX_FIFO             => 2,      -- TX fifo depth, has to be a power of two, min 1
		IO_UART1_EN                  => false,   -- implement secondary universal asynchronous receiver/transmitter (UART1)?
		IO_UART1_RX_FIFO             => 1,      -- RX fifo depth, has to be a power of two, min 1
		IO_UART1_TX_FIFO             => 1,      -- TX fifo depth, has to be a power of two, min 1
		IO_SPI_EN                    => true,   -- implement serial peripheral interface (SPI)?
		IO_SPI_FIFO                  => 2,      -- SPI RTX fifo depth, has to be zero or a power of two
		IO_TWI_EN                    => false,   -- implement two-wire interface (TWI)?
		IO_PWM_NUM_CH                => 0,      -- number of PWM channels to implement (0..60), 0 = disabled
		IO_WDT_EN                    => false,   -- implement watch dog timer (WDT)?
		IO_TRNG_EN                   => false,  -- implement true random number generator (TRNG)?
		IO_TRNG_FIFO                 => 1,      -- TRNG fifo depth, has to be a power of two, min 1
		IO_CFS_EN                    => false,  -- implement custom functions subsystem (CFS)?
		IO_CFS_CONFIG                => x"00000000", -- custom CFS configuration generic
		IO_CFS_IN_SIZE               => 32,    -- size of CFS input conduit in bits
		IO_CFS_OUT_SIZE              => 32,    -- size of CFS output conduit in bits
		IO_NEOLED_EN                 => false,   -- implement NeoPixel-compatible smart LED interface (NEOLED)?
		IO_GPTMR_EN                  => false,  -- implement general purpose timer (GPTMR)?
		IO_XIP_EN                    => false,  -- implement execute in place module (XIP)?
		IO_ONEWIRE_EN                => false   -- implement 1-wire interface (ONEWIRE)?
	)
	port map (
		-- Global control --
		clk_i       => pll_clk,       -- global clock, rising edge
		rstn_i      => pll_rstn,      -- global reset, low-active, async

		-- Custom Functions Subsystem IO (available if IO_CFS_EN = true) --
		cfs_in_i => (others => '0'),  -- custom inputs

		-- GPIO (available if IO_GPIO_EN = true) --
		gpio_o      => gpio_o_int,      -- parallel output

		-- primary UART0 (available if IO_UART0_EN = true) --
		uart0_txd_o => uart_tx, -- UART0 send data
		uart0_rxd_i => uart_rx, -- UART0 receive data

		-- SPI (available if IO_SPI_EN = true) --
		spi_sck_o   => spi_sck_int,   -- SPI serial clock
		spi_sdo_o   => spi_mosi_int,   -- controller data out, peripheral data in
		spi_sdi_i   => spi_miso_int,   -- controller data in, peripheral data out
		spi_csn_o   => spi_csn_int   -- SPI CS
	);

	-- Replicate SPI ports
	spi_mosi <= spi_mosi_int;
	flash_mosi <= spi_mosi_int;
	spi_sck <= spi_sck_int;
	flash_sck <= spi_sck_int;
	spi_miso_int <= flash_miso when spi_csn_int(0) = '0' else spi_miso;

	spi_csn <= spi_csn_int(spi_csn'range);

end architecture;

I believe the VHDL is not wrong (though I wouldn't swear on it!), but of course, I've also synthesized a similar design without the second SPI port, just the Flash-SPI one. Also, I believe that code and pinout is fine because I can look at FLASH SPI signals using an oscilloscope, and I can see these three images:

1.- Chip Select and Clock: both look fine

2.- Chip Select and MOSI: both look fine

3.- Chip Select and MISO: CS is ok, but... there's nothing on MISO. Even if I change oscilloscope trigger to CH2 falling edge, it won't trigger, MISO is always high

Even if my HDL code was wrong, specifically the MISO multiplexing, the flash should be answering to the WEL enable/disable commands that the bootloader is trying to perform to check communication with the flash. I added some extra debugging on bootloader to see what was happening:

spi_flash_read_status() is always returning 0x000000ff (all ones, which matches what I see on the oscilloscope).

Of course, I also tried to remove the second SPI port without any success, so I realy think I could in fact swear that the HDL code is right, but communication with the flash is still failing.

On the other hand, when I program a new bitfile using iceprog, I do see the process of flashing the chip. So... maybe this isn't the same port that the FPGA is connected to? That's the only explanation, but that's not what I understand from the schematic as J15 and J16 do exist by default and it seems that they are indeed connecting MISO/MOSI signals from the flash directly to the pins I defined above on constraints file.

So, right now I'm quite lost. What can I do to further investigate this? Does anyone have any idea on what's happening?

[stnolting]

Hey there!

I am implementing a system using the iCEBreaker board.

I just got an iCEBreaker board myself and I am implementing a NEORV32-based SoC there as well - so, perfect timing! 😅

I believe to have properly set constraints:

I think this is fine. Here is my SPI pin mapping (I am using Lattice Radiant so the pin mapping file looks a little bit different):

## SPI - on-board flash
ldc_set_location -site {14} [get_ports flash_sdo_o]
ldc_set_location -site {15} [get_ports flash_sck_o]
ldc_set_location -site {16} [get_ports flash_csn_o]
ldc_set_location -site {17} [get_ports flash_sdi_i]

But maybe my VHDL code is wrong somehow, because there's a little quirk about my design: I need SPI access to both the FLASH and also to an external device connected via one of the PMODs, so I had to multiplex some SPI signals out of my design:

I am also "multiplexing" the SPI module to be used with the on-board flash and a peripheral connected via one of the PMOD ports. Your code looks fine, too. Anyway, here is mine:

  -- SPI --
  flash_sck_o <= con_spi_sck;
  pmod_sck_o  <= con_spi_sck;

  flash_sdo_o <= con_spi_sdo;
  pmod_sdo_o  <= con_spi_sdo;

  flash_csn_o <= con_spi_csn(0);
  pmod_csn_o  <= con_spi_csn(1);

  con_spi_sdi <= flash_sdi_i when (con_spi_csn(0) = '0') else pmod_sdi_i;

I believe the VHDL is not wrong (though I wouldn't swear on it!), but of course, I've also synthesized a similar design without the second SPI port, just the Flash-SPI one. Also, I believe that code and pinout is fine because I can look at FLASH SPI signals using an oscilloscope, and I can see these three images:

These images were captured while the NEORV32 bootloader tries to load an executable, right? Did you connect your scope to the Flash SPI pins?

In general, the SPI transactions look pretty good. But it is strange that there is no output from the flash at all. Maybe this pin/signal is tied to a fixed level by the FPGA itself (so a problem caused by the synthesis tool).

Using the sysCONFIG pins (the SPI pins the FPGA uses to fetch the bitstream) as GPIOs might be a bit complicated... Maybe your synthesis toolchain does not support that?! Could you try building a bitstream using Lattice Radiant?

[agamez]

Hi Stephan!

These images were captured while the NEORV32 bootloader tries to load an executable, right?

Yes, there is no image loaded on the flash itself, but those images are the transactions that happen when I press 'l: Load from flash' on the bootloader. By the way, the countdown from 8 doesn't always appear... is it related? Have I done something wrong with the bootloader and that's the culprit of my issues? It looks like I need to enable MTIME, and that enabled the counting down, but didn't get flash to work (as expected, but who knows what else I'm missing?)
Oh, and just to give out all the information I can... I'm running neorv32 v1.8.0. That was the latest tag I found when I started toying around with neorv32 and I didn't want to keep jumping versions all the time.

Did you connect your scope to the Flash SPI pins?

Indeed, I taped directly to Flash SPI pins below on the image. I left one channel always connected to CS pin so I could trigger the oscilloscope with that channel. My board connects to another piece of hardware using PMOD1A.

In general, the SPI transactions look pretty good. But it is strange that there is no output from the flash at all. Maybe this pin/signal is tied to a fixed level by the FPGA itself (so a problem caused by the synthesis tool).

Using the sysCONFIG pins (the SPI pins the FPGA uses to fetch the bitstream) as GPIOs might be a bit complicated... Maybe your synthesis toolchain does not support that?! Could you try building a bitstream using Lattice Radiant?

Oh, that would have been great news! However, I've just tested this on Radiant with the same results: there's nothing I can see coming out of the flash. But the flash itself works because I can see that pin sending data when the FPGA programs itself on power-up. There's something fishy going on here... but I don't know what! and I don't know why, I can't get Radiant to build a bitfile that launches neorv32 properly. So... I can't confirm yet that this is a toolchain issue because the official toolchain isn't working for me yet, it seems like I have to change something but I still don't know what.

[agamez]

Hi again!

We got our answer! I asked on 1BitSquared discord and @gatecat was very kind to read the whole issue and solve it in one go:

there's nothing special required to use the iCE40 flash pins as user IO, neither with Yosys/nextpnr nor with iCEcube or Radiant
however the iCE40 does put the flash into deep sleep mode by default after power-up
this needs either a command sent to wake it (0xAB from memory) or passing "-s" to icepack (or changing the equivalent setting in radiant) to disable this behaviour
(for the record, the relevant documentation on the hw side is https://www.latticesemi.com/~/media/LatticeSemi/Documents/ApplicationNotes/IK/FPGA-TN-02001-30-iCE40-Programming-Configuration.pdf?document_id=46502 page 20, although it omits that this is also the default)

I tested the "-s" parameter on icepack and it solved the issue. I've already stored and loaded my program from FLASH automatically after the 8s timeout.

Now, this is one way to do it, but maybe it's better for the bootloader to wake the eeprom and put it to sleep back again? It probably doesn't need to test first if it's sleeping, just wake it up. Or maybe we don't want to put it to sleep again.

I don't mind writing a patch for either case, but I'd like to know what's your opinion. I believe the right thing would be to wake it, regardless of its state on boot, and then put it to sleep again. It should be the user application code the one who wakes it up when/if needed and we would save some mW of power.

[stnolting]

I was not aware of this... so thank you very much for sharing your findings! 👍

Now, this is one way to do it, but maybe it's better for the bootloader to wake the eeprom and put it to sleep back again? It probably doesn't need to test first if it's sleeping, just wake it up. Or maybe we don't want to put it to sleep again.

I absolutely agree! I think command 0xAB is the standard command to get the flash out of sleep mode. Maybe we could issue that command at the beginning of the spi_flash_check function? 🤔

neorv32/sw/bootloader/bootloader.c

Line 749 in 7a2386b

int spi_flash_check(void) {

I think it should be sufficient to wake up the flash by the bootloader. Putting it back to sleep is also a nice feature, but I cannot find a standard command for that. Furthermore, we should keep the bootloader size as minimal as possible 😉

[stnolting]

For the records: implemented in #552