S4810Init < Computing

Computing Web>S4810Init (29 Apr 2013, _47C_61UK_47O_61eScience_47OU_61Birmingham_47L_61ParticlePhysics_47CN_61lawrence_32lowe? ) (raw view)
EditAttach
---+ Initialising a Dell Force10 S4810 switch

A S4810 switch does not come configured, so needs to be configured even to act as a layer 2 switch. To quote the manual, initially interfaces are disabled and in Layer 3 mode.
---++ Connecting the serial cable to the console management port

The S4810 can be configured initially via its serial port. (That re-configuration can include enabling the IP management port, so that subsequently it can be managed via that port instead). So I (LSL) fitted the supplied serial cable, RS232 9-pin to RJ45 plug, to the S4810 serial RS232 console management port, which is the upper RJ45 port on the right of the front face of the unit. I connected the other end to a suitable serial terminal or terminal emulator, with RS232 settings 9600,8,1,N.
---++ Setting the switch hostname, and date and time

First we can set the hostname, which appears in prompts at the keyboard, and the timezone and corresponding time and date in the software clock, and then send that information to the hardware clock ("calendar") so that it survives a re-boot.

Note added later: setting a hostname can be misleading within a stack. If the switch becomes part of a stack, not the master, then it adopts the hostname of the stack master, _and this is written into the startup-config file and thus retained even if the power is removed from the switch, the stacking cable removed, and then power restored_. This could be a bug (as of version 8.3.10.2). Consequently where switches might be stacked, it could be more sense not to set a hostname, but to rely on the stack-unit number (visible in the first page of show running-config), or the assigned management IP address, as a means of verification of which switch you are managing. Alternatively, keep a back-up of the original configuration, see later section, so that it can be restored if de-stacked.
<verbatim>> enable
# config
(conf)# hostname mychosenname
(conf)# clock timezone UTC 0
(conf)# exit
# clock set hh:mm:ss dd month yyyy 
# clock update-calendar
# show clock
# show calendar 
</verbatim>

---++ Stacking preliminaries

If a particular S4810 might be part of a stack, sooner or later, some preliminary settings are best done now. You can set the stack-number to some value 0-11, in place of the stand-alone default of 0. This avoids the stack master dynamically re-assigning the stack-number of a joined stack-member if there is a number conflict/clash. If the switches have some sort of external label number anyway, this would be a good choice, so for my switch number 3:
<verbatim># show system stack-unit 0
# stack-unit 0 renumber 3
(prompts and reboots)
# show system stack-unit 3
# show system brief 
</verbatim>

NOTE: changing the stack-unit number causes a reload of the firmware, ie after the renumber command above, even if the switch is currently standalone!

NOTE: the stack-unit number appears to be recorded in flash memory in a hidden location, and not in the startup-config file.
---++ Layer 2 configuration

After entering an enable command to get a hash prompt, enter a *show interfaces status* command in order to list the ports you have, including their slot number: the digit(s) before the slash in the port's identifier. This slot number may be zero, or set to the stack-unit number if you have renumbered as in the section above (Stacking preliminaries). Then configure the ports for layer 2. Here is an example with slot number of 3:
<verbatim>> enable
# show interfaces status 
# config
(conf)# interface range tengig 3/0 - 47
(conf-if-range-te-3/0-47)# switchport
(conf-if-range-te-3/0-47)# no shutdown
(conf-if-range-te-3/0-47)# exit
(conf)# exit
# write mem      (or reload, and reply yes to Save question)</verbatim>

---++ Configuring an IP address on the management ethernet port

The management ethernet port is located below the console RJ45 port. It supports 100Mb/s connections (and probably others). I needed to configure it so that I could download new software, and so that I could later remotely login to the switch via a connection to that port. So it needed to be configured with an IP address and to be cabled to the corresponding network. The IP address and network can of course be any network suitable for administration: not necessarily related to the network handled by the switch. Here's an example with a slot-number of 3 (see previous section about slot-numbers):
<verbatim># enable 
# config
(conf)# interface man 3/0 
(conf-if-ma-3/0)# ip address a.b.c.d/24
(conf-if-ma-3/0)# show config
(conf-if-ma-3/0)# exit
(conf)# exit
# write mem      (or reload, and reply yes to Save question)</verbatim>

---++ Updating the version of the FTOS software

The Force10 portal provides downloadable images for the latest FTOS and Boot firmware. My units arrived with version 8.3.7.0, which was quite old. Visit the Dell Force10 portal, currently http://www.force10networks.com/, click _Service and Support_, and then _S-Series Downloads_ on the left, apply for an ID if you haven't got one already; when you have an ID, click login and do the login, and then click _Software Center_ near the top, and you should see suitable download images.

So I downloaded the FTOS and boot images to a local server machine. This has a ftp server running on it, for local access. The management port had been configured with an IP address (see above), and was cabled to the same network that the server machine was on. On the S4810 console port, I entered:
<pre><verbatim>$ enable 
# upgrade system ftp: A: 
Address or name of remote host: p.q.r.s 
Source filename: FTOS-SE-8.3.10.2.bin     (for example)
User name to login remote host: myid
Password to login remote host: mypass
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Erasing Sseries Primary Image, please wait
........................... (about 20 lines of these)
23955875 bytes successfully copied (for example)
# reload            (or continue on to upgrade the Boot Code)</verbatim></pre>

---++ Upgrading the Boot Code
 <verbatim># upgrade boot ftp:
Address or name of remote host: p.q.r.s 
Source filename: U-boot.1.2.0.2.bin     (for example)
User name to login remote host: myid
Password to login remote host: mypass
Erasing Sseries BootImage ......, please wait
................. 524528 bytes successfully copied
# reload
 </verbatim>

---++ Remote call-in to the management port

This assumes that the management port has been given an IP address (see above) and is cabled to a corresponding network.

If you telnet from a computer to the management port IP address, and you get the message "Error: login failure", then you haven't configured a username yet. Also note it seems to take about 90 seconds after the S4810 appears to be up before it will respond to a telnet call: for me it's not immediately after the system is apparently up.

You need to configure at least one username as follows. If you want any remotely-logged-in username to be able to enter the _enable_ command get into exec mode, then you also need the second command below to give the enable command itself a protective password: that password then also applies during sessions on the RS232 console.
<verbatim>(conf)# username someID secret somepass
(conf)# enable secret somepass2</verbatim>

---++ Saving the configuration

The loadable configuration can be seen using *show startup-config*. There are two sensible places for saving the configuration: one is in a back-up file in the local switch flash-memory filesystem, and the other is on some other backed-up storage on a file-server. Saving the configuration is useful particularly if the current configuration will be over-written by stacking and you want to be able to return at some future point to the original configuration. Here's a local and remote save:
<verbatim># copy startup-config  startup-config.unit3.YYYYmmdd
# copy startup-config  ftp://ii.pp.qq.rr/startup-config.unit3.YYYYmmd</verbatim>

---++ Restoring a configuration

To restore a configuration, simply copy the startup-config file back using similar commands to the above. Then, if you changed the stack-port(s) or stack-unit number since saving that configuration, then do appropriate commands to return it to how it was. This is necessary because this stack information appears to be recorded in flash in a hidden location and not in the startup-config file (unconfirmed in current documentation). Renumbering will ask to reboot the switch; if not renumbering, do a reload command to reboot the switch.
---++ Stacking implementation

Following setting the stack-unit number (see Stacking preliminaries section), it's necessary to designate what physical ports are going to contain the cable(s) for the stacking. These can either be from the 40 Gbit ports (namely, ports 48, 52, 56, and 60, which are referred to as stack-group numbers 12,13,14, and 15), or from the 10 Gbit ports in bunches of 4 (so for example ports 44-47 are referred to as stack-group 11). The cables used for use with stacking are not special but of course the appropriately-labelled cable(s) have to be used for a port: 40 Gbit cable or 10 Gbit cable. In the following, on stack-unit number 2, we're going to use the port labelled 60:
<verbatim># stack-unit 2 stack-group 15
# show system stack-ports 
</verbatim>

On the other switch, say stack-unit number 1, again specify the physical port where the other end of the cable will go: for example, again the port labelled 60:
<verbatim># stack-unit 1 stack-group 15
# show system stack-ports 
</verbatim>

In both cases, the switch will require a reload for the stack-port(s) to take effect. You might want to do this before the cable is inserted at both ends.

Having got both switches thus ready for stacking, we are nearly ready to insert the stacking cable. At this point you should consider which physical switch you want to end up as master. The one which is not master will reboot when connected in order to become a standby rather than a stand-alone master. The rule is that, if stacking priorities of the switches are the same, then the switch with the highest chassis MAC address will become master. (For me, the switch hostnames and their MAC addresses are both in the same order). If this is not desired, you can set the stacking priority of the preferred master to be higher than the default of 0, for example:
<verbatim># config
(conf)# stack-unit 1 priority 5</verbatim>

You can specify a number between 1 and 14. If at any point you want to return the priority to the original default of 0, then you have to prefix the command above by "no " to negate it, rather than specifying 0 as the priority.

Then insert the stacking-cable. The switch which is not master will then reboot. When it comes back up, it no longer reports its own messages, which now all appear on the master. If contacted via the serial port, it prompts Login rather than just waking up, and if you login, the prompt is masterhostname(standby). It does not appear to respond to its IP management port. The master on the other hand responds to its serial port and IP management port, and the status of either switch can be got via the one interface.
---++ Disconnecting a stack

If the stacking cable is removed between two stacked switches, they go back to being separate switches. The software does not reload. However, note that although both switches retain their stack-unit number, the subsidiary switch keeps its adopted configuration including its adopted hostname (as revealed in a prompt), and loses its original hostname: it does not return to the one set using the "hostname" command when it was a stand-alone master. This could be a source of confusion and looks like a bug rather than a feature to me. A solution is to save the configuration and then restore it when required, a trivial operation (see previous section: Saving the configuration).

Note at random: there is also a movemanagement command to transfer the manager to a different switch in the stack.
---++ Port-channels (bonded/aggregated ports)

If you have bonded the host interfaces on the server to get greater overall throughput, then you will want to create the corresponding setup on the S4810 switch.

Notice that as usual even if the ports have gigabit modules inserted in them, you still refer to the ports as tengig ports.

First you have to remove the switchport status of the interfaces you are going to use for port-channels. For me, this was ports 12-19. Then for each port-channel, declare its requirements and add member ports to it. The port-channel number you specify on the interface port-channel command is fairly arbitrary (1-128), but for sanity I have used the lowest-numbered port of each bond as the port-channel number.
<verbatim>config#interface range tengig 0/12 - 19
(conf-if-range-te-0/12-19)#no switchport
(conf-if-range-te-0/12-19)#no shutdown
(conf-if-range-te-0/12-19)#exit

(conf)#int port-channel 12
(conf-if-po-12)#switchport
(conf-if-po-12)#no shutdown
(conf-if-po-12)#channel-member tengigabitethernet 0/12
(conf-if-po-12)#channel-member tengigabitethernet 0/13
(conf-if-po-12)#exit 

(conf)#int port-channel 14
..............................................
(conf)#int port-channel 18
(conf-if-po-18)#switchport
(conf-if-po-18)#no shutdown
(conf-if-po-18)#channel-member tengigabitethernet 0/18
(conf-if-po-18)#channel-member tengigabitethernet 0/19
(conf-if-po-18)#exit
(conf)# exit
# show int port brief 

 </verbatim>

---++ Useful commands

*show version* show the software version number, the uptime, and also a summary count of the different ports on the unit.

*show env* shows the status of fans and power supplies and temperatures on the unit.

*show interfaces status* shows a one-line summary for each port on the switch unit (except those configured as stack-ports or channel-ports).

*show interfaces port-channel brief* shows a few lines per (bonded) port-channel

*show system* shows a bunch of lines per connected stack-unit.

*show system brief* shows one line per connected stack-unit and then one line for each of their power supplies and fan units.

*show system stack-unit n* shows the specified stack-unit status (if it is this switch or stacked with this switch).

*show system stack-ports* shows which ports on the switch are configured as ports for stacking cables.

*show startup-config* shows the configuration settings permanently stored in the startup-config file in flash memory.

*show running-config* shows the configuration settings currently in use in RAM.

*dir* shows a list of the files/directories in the flash file-system, including the startup-config file.
---++ Insertion/removal of 1GbE and 10GbE port modules

The S4810 comes with 48 SFP+ direct ports, numbered 0 to 47, arranged in an upper and lower row. Plug modules can be inserted into any of these ports.

Note to myself: remember that the 1GbE modules only support 1 Gb/s: they do not support 10Mb/s or 100 Mb/s. I've wasted several minutes on two separate occasions wondering why my netbook-port, or wall-ports with a 100 Mb/s interface, doesn't cause the LEDs to light-up. As they say, the clue is in the name.

Insert a 1GbE module such that the yellow plastic slider is towards the imaginary line separating the upper and lower row of sockets. It would be hard to insert it otherwise, but this saves looking. To remove a 1GbE module, position the thin metal tag outward and pull gently on it. This unlocks the module so it is easily withdrawn.

Insert a 10GbE fibre module so that the decorated arm is on top for the upper row. To remove a 10GbE fibre module, swing down the rocker arm and pull gently on it.

Insert a copper cable (10GbE or 40GbE) by just pushing it in. To remove, pull on the blue tag. If it sticks, just keep the blue tag taut and wiggle the plug out.
---++ Status

| Name | hostname set | software upgrade | config man port | enable pass | lsl/pass LAN login | ppbxxx LAN login | stack-unit renumbered | stack-unit priority | stack-port configured |
| epsw30 | yes 20121031 | yes | yes | yes | yes | yes | leave as 0 | leave as 0 | no |
| epsw31 | yes 20120531 | yes | yes 20120531 | yes | yes 20120531 | yes 20120531 | Set to 1 (20130130) | leave as 0 | 15 (port 60) |
| epsw32 | yes 20120530 | yes | yes 20120530 | yes | yes 20120530 | yes 20120530 | not yet | leave as 0 | not yet |
| epsw33 | yes 20121029 | yes | yes | yes | yes | yes | Set to 3 (20130122) | leave as 0 | 15 (port 60) |
| epsw34 | yes 20121030 | yes | yes | yes | yes | yes | Set to 4 (20130122) | leave as 0 | 15 (port 60) |
| epsw35 | yes 20120528 | yes | yes 20120521 | yes | yes 20120521 | maybe | not yet | leave as 0 | no |

Document started Main.LawrenceLowe - 17 May 2012
Topic revision: r18 - 29 Apr 2013 - _47C_61UK_47O_61eScience_47OU_61Birmingham_47L_61ParticlePhysics_47CN_61lawrence_32lowe?
Computing
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback