Home

This is the official wiki for the IPMC Framework maintained by the University of Wisconsin-Madison and it is known to be INCOMPLETE.

For questions related to the framework or documentation please reach out to:

• Marcelo Vicente (marcelo.vicente AT cern.ch)
• Jesra Tikalsky (jtikalsky AT hep.wisc.edu)

We will either reply to questions directly or clarify/add in the documentation.

Index

• Index
• Introduction
• Requirements
• Git Repository Structure
  • New application specific project
• Fresh checkout
• Create a new branch for the application
  • Open Vivado for firmware editing
  • Open XSDK for framework customization
• Customization
  • Firmware
  • Software
    • Drivers and services
    • IPMI and payload customization
• Q&A
• Missing items in documentation

Introduction

The ZYNQ-IPMC Framework is a set of high-level drivers, services, libraries and utilities based on FreeRTOS and C++11. It is targeted to run on the ZYNQ-IPMC hardware platform powered by a Xilinx Zynq SoC which has both a FPGA and an ARM CPU in the same package connected by AXI interfaces. The framework leverages several automation tools maintained by Xilinx which simplifies the design process.

The primary use case of the ZYNQ-IPMC hardware and framework is to be used as an IPMI controller in Advanced TCA (ATCA) blades as part of the PICMG specification. In a typical application, the ZYNQ-IPMC will:

• Communicate with the crate's Shelf Manager via IPMB;
• Control power-on/off and proper sequencing;
• Read sensors from the ATCA blade (e.g. temperature, voltage, current, etc.);
• Report sensor values to the Shelf Manager;
• Detect and react to any fault condition at sub-millisecond reaction times;
• Expose a Xilinx Virtual Cable (XVC) to any wired FPGA/SoC via Ethernet.

Requirements

• Linux (likely to work on Windows with Cygwin).
• Xilinx Vivado 2018.2 with XSDK installed.

Git Repository Structure

All core and common components of the IPMC Framework are set as submodules while application specific implementations are independent git repositories that reference the framework submodules.

The following three submodules form the framework:

• IPMC Custom Firmware IP repository
• First Stage Boot Loader (FSBL)
• Common Framework Code (drivers, services & libraries)

In case of updates, submodules with newer tags can pulled from the main IPMC git repository without affecting application specific code. If changes or additions are required to code in submodules, it is recommended to request merges to the main repository since they are likely to be of interest to the rest of the IPMI community.

For an example of the project structure check the cdb branch. This project will be used throughout this documentation as reference.

New application specific project

It is highly recommended to fork and clone one pre-existing project and start from there. The recommended starting branch would be cdb or fallback.

Fresh checkout

• Run git clone git@github.com:uwcms/IPMC.git --recursive for a clean checkout.

If a different branch is required after checkout:

• In the project folder run git checkout [branch name] to change to a different branch.
• Run git submodule update to properly use the project submodules.

Create a new branch for the application

• Run git branch [new branch name] to create a new branch for the project.
  • Tip: Use the project acronym for the branch name for example.
• Keep track of that branch by running git checkout [new branch name].

Open Vivado for firmware editing

• The Vivado project (.xpr) is located at Vivado/ipmc_zynq_vivado.xpr in the project folder.
• In the project folder run vivado Vivado/ipmc_zynq_vivado.xpr to open the project.
  • Don't forget to source Vivado if running on Linux, normally by running source /opt/Xilinx/Vivado/2018.2/settings64.sh.

Open XSDK for framework customization

It is easiest to lunch XSDK from within Vivado so these instructions assume that the project is already open.

• In Vivado, go to File -> Launch SDK from the Menu Bar.
• A dialog will show up, leave the defaults and press OK.

If this is a fresh checkout additional steps are required to import the software projects:

• In XSDK, go to File -> Import -> General -> Existing Projects into Workspace from the Menu Bar.
• The import dialog will appear. Press the Browse button and then press OK to select the default folder.
• XSDK will automatically detect the existing projects. Make sure at least fsbl and IPMC are checked and press Finish.

Customization

Both the Programmable Logic (PL) and Programmable System (PS) of the ZYNQ need to be customized separately and will heavily depend on the target application. This section covers the recommended guidelines on how to do it properly.

Firmware

Firmware customization is done using Vivado Design Automation and IP Integrator for simplicity. Watching the video in this link is highly recommended to get familiarized with these tools.

Customization of the firmware is done in 5 steps:

1. Add AXI IPs from the Xilinx IP catalog or from the IPMC Framework to the IP Integrator.
   • Most standard Xilinx IPs (e.g. IIC, SPI, UART, etc.) have driver support.
2. Expand and connect the added IPs to the main AXI Interconnect and wire the rest of the signals.
   • Signals that will be connected to pins must be made external by pressing right-click on the interface and then Make External.
   • It is recommended to use and connect interrupts if the IP supports it.
   • The pin name can be changed in the properties.
3. Synthetize and open the synthetized design. From here go to Layout -> IO Planning in the Menu Bar.
   • Pin mapping will show at the bottom, make sure that all pins are mapped and fixed.
   • Save the changes (e.g. Ctrl+S).
4. Run implementation and Generate Bitstream.
5. Export the hardware to XSDK by going to File -> Export -> Export Hardware in the Menu Bar.
   • Make sure the bitfile is included in the export.
   • If XSDK is open the changes will be automatically detected.

Software

There are two major sets of modifications required per application: application specific (e.g. drivers and services) that depend on the firmware, and IPMI related settings. Both are done in different files or set of files.

Drivers and services

Firmware peripherals that are supported by the framework have their software drivers located here. These are first initialized void driverInit() and then, depending on the specific application requirements, can be used in high-level services that are located here. Services are initialize in void serviceInit(). Both functions are located in ipmc.cpp.

By default, the framework will automatically initialize several application agnostic drivers and services, including:

• Watchdog Timer
• Serial Console
• Persistent Storage
• Logging
• IPMI

The following list provides guidelines on how to modify the software:

1. Change the framework settings at config/zynqipmc_config.h, if any.
   • For an application that won't need IPMI in the beginning it is recommended to comment out ENABLE_IPMI to disable it.
   • Of the code is crashing and the watchdog timer is rebooting the IPMC then it is recommended to comment out ENABLE_WATCHDOGTIMER to facilitate debugging.
2. Create the high level driver objects in driver_init() at ipmc.cpp.

• ADC initialization also takes place here and channel naming should be meaningful since they are used for IPMI as well.

3. Initialize the services in service_init() at ipmc.cpp.
   • Network services should be initialized in the labda callback when Network finishes initialization.

All drivers and services have inline documentation with Doxygen support and are easy to understand for a programmer with average C++ knowledge.

IPMI and payload customization

IPMI and payload configurations are separated in a few files based on their context.

1. Start by editing the content of void initFruData(..) in fru_data_init.cpp which contains blade specific information, like name, manufacturer, etc. The information here will get propagated to the shelf when it needs to know what type of blade is inserted.
2. Modify the SDR records in void initDeviceSDRs(bool) in sdr_init.cpp.

• Sensor records mcdlr and hotswap and common and shouldn't be changed.
• Sensors can be anything that is card related and monitorable.
• Only sensors that need to be reported to the shelf manager need to be declared.
• The names of the sensors are used in other portions of the code to link SDRs with ADCs. Mismatches will be shown during boot.
• The SDR Calculator tool is used to facility SDR code generation.

3. Change card behavior to sensors in board_payload_manager.cpp and board_payload_manager.cpp.

• E-Keying, power zones, hard-fault response, power pin behavior and sequencing is defined in void BoardPayloadManager::config().
• Power levels are implemented in void BoardPayloadManager::setPowerLevel(uint8_t, uint8_t) and void BoardPayloadManager::implementPowerLevel(uint8_t). For most applications only small changes are required here.

Q&A

Nothing as of yet.

Missing items in documentation

• Partition documentation in several files.
• Consider adding pictures on how to customization the firmware.
• Expand on firmware customization, show common IPMC examples.
• Make a section explaining the lingo (e.g. payload, etc.).
• Talk about bench mode operation.
• Mention that common services are already initialized outside the application specific code.