37

The SuperMicro X8SIE-F board has two dedicated LAN interfaces for the operating system (LAN1/2) and one dedicated LAN interface for IPMI.

Is it possible to configure IPMI to use one of the LAN1/2 interfaces, instead of the IPMI port? If so, what is the procedure?

Jeff Atwood
  • 13,104
  • 20
  • 75
  • 92
80skeys
  • 765
  • 2
  • 9
  • 15

7 Answers7

59

Jiri's on the right track with the three options (Dedicated, Share, Failover) for the IPMI interface. The short answer is that yes, you can use LAN1 instead of the dedicated IPMI port, and it generally works that way with the default BIOS settings. It's not possible to run the IPMI on the LAN2 interface.

Here's a more detailed description of the three options:

  • Dedicated: Always use the dedicated IPMI interface. This is the option you want if you're trying to have the simplest setup, at the expense of additional cabling.

  • Shared: Always use the LAN1 interface. This is the option you want if you're trying to reduce your cabling to each server, and understand the tradeoffs. Under the covers, there's a virtual switch in hardware that's splitting out traffic to the IPMI card from traffic to the rest of the system; the IPMI card has a separate MAC address to differentiate the traffic. On modern Supermicro boards, you can also set the IPMI traffic to run on a different VLAN from the rest of the system, so you can tag the IPMI traffic. There are some definite security implication to this design; it's not difficult for the main system to access the IPMI network, if you were trying to keep them separated. A failure of the LAN1 interface often means that you lose primary and out-of-band connectivity at the same time.

  • Failover (factory default): On boot, detect if the dedicated IPMI interface is connected. If so, use the dedicated interface, otherwise fall back to the shared LAN1. I've never found a good use for this option. As best I can tell, this setup is fundamentally flawed - I haven't tested it extensively, but I've heard reports it'll fail to detect the dedicated interface in many circumstances because the upstream switch isn't passing traffic - for example, after a power outage if the switch and system come up simultaneously, or if the switch is still blocking during the spanning tree detection. Combine this with the fact that the check only happens at boot, and it's just generally hard to control what interface you end up using.

Jeff Atwood
  • 13,104
  • 20
  • 75
  • 92
natacado
  • 3,367
  • 29
  • 27
  • 11
    Failover results in more problems than it's worth, in my opinion. There's no easy way to tell if your IPMI NIC has failed over or not. It requires a power cycle (not just a reboot) to reset the port back to the dedicated one. – Daniel Lawson Feb 21 '12 at 00:13
  • Thanks a bunch for that detailed explanation. In IPMIView I see the three options, but there's no "Help" button or explanation to describe what they do. – 80skeys Feb 21 '12 at 19:04
  • 2
    On linux, ipmitool mc reset cold mostly helps in that situation, but this can also cause strange behaviour on the host for several seconds since the kernel has to deal with the devices the BMC emulates behaving like they suddenly were ripped off the bus. Another problem with failover is that a) it is the default, b) it can unexpectedly make a dedicated sideband lan lose connectivity, c) throw the bmc straight in harms way into a DMZ. – rackandboneman May 06 '12 at 16:23
  • 2
    note that the **factory default is IPMI failover** (at least on the 5 new SuperMicro based servers I just bought) which leads to a lot of.. confusion. This is dangerous! – Jeff Atwood Oct 23 '12 at 19:23
  • Is there a way to see which option (Dedicated, Share, Failover) is in effect for the IPMI interface using a tool like `ipmitool`? – Stefan Lasiewski Mar 01 '13 at 22:40
  • 2
    @StefanLasiewski: See [this](http://blog.multiplay.co.uk/2013/06/configuring-supermicro-ipmi-interface-using-ipmitool/). – Dennis Williamson Aug 19 '13 at 21:54
  • On my 7049 chassis, it's stuck in Failover. How do I change to the other modes? – Elliott B May 03 '18 at 01:20
38

The following raw commands will surely prove useful to the next person who reads this topic. I got these directly from Supermicro support. Cheers:

To get LAN mode: ipmitool raw 0x30 0x70 0x0c 0.

To set LAN mode dedicated: ipmitool raw 0x30 0x70 0x0c 1 0.

To set LAN mode onboard/shared: ipmitool raw 0x30 0x70 0x0c 1 1.

To set LAN mode failover: ipmitool raw 0x30 0x70 0x0c 1 2.

These raw values will also work with ipmicfg (just remember to use -raw instead of raw).

After setting the required LAN mode, do not forget to do a hard reboot.

lobi
  • 1,083
  • 2
  • 15
  • 30
15

In these boards, the ipmi interface can only be configured on the 1rst LAN or in the dedicated one, and you can choose it by issuing some raw commands. My steps to enable the dedicated LAN on a X8DTU-F board are these:

#install ipmitool (this is for debian)
apt-get install ipmitool
#insert the kernel modules needed for ipmi
modprobe ipmi_devintf
modprobe ipmi_si
modprobe ipmi_msghandler
#get the current mode (01 00 is dedicated mode)
ipmitool raw 0x30 0x70 0x0c 0
#send the raw command to enable dedicated lan
ipmitool raw  0x30 0x70 0xc 1 1 0

You can now use ipmitool to assign an ipaddress/netmask to the interface, and start working with it normally.

If you want to know all the comands in raw mode for the supermicro ipmi, there's a list here

aseques
  • 718
  • 4
  • 12
  • 28
  • 1
    Most useful answer here, especially the link to the SMC FAQ, which makes it clear that different motherboards use slightly different versions of this command. – Nemo Oct 11 '17 at 00:16
14

I figured it out. The way it works is pretty hokey, but here it is:

It depends on whether there is a network cable plugged into the dedicated IPMI port, before the server receives power. If a cable is present, then IPMI is automatically run on that port. If there is no cable in that port, then IPMI is automatically assigned to the LAN1 port (shared with the operating system).

These settings are not persistent through power cycles unless you make them permanent using IPMI View tool > BMC Settings. If you do not do this, then every time the server gets power cycled, it will test the cable again and reassign IPMI accordingly.

Skyhawk
  • 14,200
  • 4
  • 53
  • 95
80skeys
  • 765
  • 2
  • 9
  • 15
5

You can also use ipmitool to set the lanport:

sudo ipmiutil smcoem lanport dedicated

Then check that your config is right:

sudo ipmiutil lan -e

You should see the line:

SuperMicro Lan Interface  :  00     :  Dedicated

Cheers!

Lacek
  • 7,233
  • 24
  • 28
Bud Hann
  • 51
  • 1
  • 1
2

I have X8SIL-F-B and if you connect to BMC via lan with browser, you can see in 'Configuration - Network' that for LAN interface there are more options (dedicate, share, failover).

But I haven't tested it.

jirib
  • 1,240
  • 8
  • 15
1

Here is another usefull Supermicro IPMI tool for users having issues: IPMICFG This has helped me out of a few jams before.

JohnRB
  • 11
  • 2