Hello,
I need help with a very strange problem with Intel E1000 NIC; generating
heavy UDP traffic I can reproducibly cause the driver and/or card to stop
transmitting/receiving; the meassge I get is
NETDEV WATCHDOG: eth1: transmit timed out
and sometimes later it recovers with
e1000: eth1 NIC Link is Up 1000 Mbps Full Duplex
The workload is a video streaming server generating lots of UDP packets of
varying sizes, typically many "undersized" packets (typically 480MBit/s at
80000-90000 pkts/s); very few UDP packets are received (around 400 pkt/s),
and very little TCP traffic (around 20-30 packets/s).
The problem occurs quite regularly when transmitting more than about
450MBit/s; increasing the tx rate greatly increases the frequency with
which this problem appears.
The problem does _not_ occur at all when simply transmitting mtu-sized
udp packets, where I can easily reach >900MBit/s without any problem.
Other notes:
- disabling sg, rx/tx checksumming (ethtool) does not make a difference
- disabling tso (I read this somewhere as a tip) does not apply as this
is kernel version 2.4.24 which AFAIK does not support tso (really?)
- NAPI is disabled as well
- somewhere I read to disable ACPI; however this is a ppc and thus does
not apply
- #define'ing DBG to 1 in the driver sources produces lots of output, but
unfortunately nothing useful before the "transmit timed out" message (log
attached anyway)
The system is a dual PowerMac G4, and the NIC identifies itself as:
10:15.0 Ethernet controller: Intel Corp. 82543GC Gigabit Ethernet
Controller (Copper) (rev 02)
Subsystem: Intel Corp. PRO/1000 T Server Adapter
Flags: bus master, 66Mhz, medium devsel, latency 16, IRQ 58
Memory at 800c0000 (32-bit, non-prefetchable) [size=128K]
Memory at 800a0000 (32-bit, non-prefetchable) [size=64K]
Expansion ROM at 80090000 [disabled] [size=64K]
Capabilities: [dc] Power Management version 2
Driver version is:
Intel(R) PRO/1000 Network Driver - version 5.2.20-k1
Copyright (c) 1999-2003 Intel Corporation.
I appreciate any help as this is really a show-stopper for me; I can run
tests and produce debugging output whatever you like
Thanks and best regards
Helge
Feb 9 19:49:07 testmac kernel: e1000_check_for_link
Feb 9 19:49:07 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:07 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:07 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:07 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:07 testmac kernel: e1000_update_adaptive
Feb 9 19:49:09 testmac kernel: e1000_check_for_link
Feb 9 19:49:09 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:09 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:09 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:09 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:09 testmac kernel: e1000_update_adaptive
Feb 9 19:49:10 testmac kernel: NETDEV WATCHDOG: eth1: transmit timed out
Feb 9 19:49:10 testmac kernel: e1000_reset_hw
Feb 9 19:49:10 testmac kernel: Masking off all interrupts
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Issuing a global reset to MAC
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Masking off all interrupts
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_init_hw
Feb 9 19:49:10 testmac kernel: e1000_id_led_init
Feb 9 19:49:10 testmac kernel: e1000_set_media_type
Feb 9 19:49:10 testmac kernel: Initializing the IEEE VLAN
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_init_rx_addrs
Feb 9 19:49:10 testmac kernel: Programming MAC Address into RAR[0]
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Clearing RAR[1-15]
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Zeroing the MTA
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_setup_link
Feb 9 19:49:10 testmac kernel: e1000_read_eeprom
Feb 9 19:49:10 testmac kernel: e1000_acquire_eeprom
Feb 9 19:49:10 testmac kernel: e1000_release_eeprom
Feb 9 19:49:10 testmac kernel: After fix-ups FlowControl is now = 3
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_setup_copper_link
Feb 9 19:49:10 testmac kernel: e1000_phy_hw_reset
Feb 9 19:49:10 testmac kernel: Resetting Phy...
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_detect_gig_phy
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_set_phy_type
Feb 9 19:49:10 testmac kernel: PHY ID 0x1410C50 detected
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Phy ID = 1410c50
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_phy_reset
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg_ex
Feb 9 19:49:10 testmac kernel: Reconfiguring auto-neg advertisement params
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_phy_setup_autoneg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: autoneg_advertised 2f
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Advertise 10mb Half duplex
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Advertise 10mb Full duplex
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Advertise 100mb Half duplex
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Advertise 100mb Full duplex
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Advertise 1000mb Full duplex
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg_ex
Feb 9 19:49:10 testmac kernel: Auto-Neg Advertising de1
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg_ex
Feb 9 19:49:10 testmac kernel: Restarting Auto-Neg
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_write_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: Unable to establish link!!!
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: Initializing the Flow Control address, type and timer regs
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_reset_adaptive
Feb 9 19:49:10 testmac kernel: e1000_phy_get_info
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: PHY info is only valid if link is up
Feb 9 19:49:10 testmac kernel:
Feb 9 19:49:10 testmac kernel: e1000_config_collision_dist
Feb 9 19:49:10 testmac kernel: e1000_check_for_link
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_update_adaptive
Feb 9 19:49:10 testmac kernel: e1000_check_for_link
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_update_adaptive
Feb 9 19:49:10 testmac kernel: e1000_check_for_link
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_update_adaptive
Feb 9 19:49:10 testmac kernel: e1000_check_for_link
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:10 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:10 testmac kernel: e1000_update_adaptive
Feb 9 19:49:12 testmac kernel: e1000_check_for_link
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_update_adaptive
Feb 9 19:49:12 testmac kernel: e1000_check_for_link
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_update_adaptive
Feb 9 19:49:12 testmac kernel: e1000_check_for_link
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:12 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:12 testmac kernel: e1000_update_adaptive
Feb 9 19:49:14 testmac kernel: e1000_check_for_link
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_update_adaptive
Feb 9 19:49:14 testmac kernel: e1000_check_for_link
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_check_downshift
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_config_dsp_after_link_change
Feb 9 19:49:14 testmac kernel: e1000_config_mac_to_phy
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_config_collision_dist
Feb 9 19:49:14 testmac kernel: e1000_config_fc_after_link_up
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: Flow Control = FULL.^M
Feb 9 19:49:14 testmac kernel:
Feb 9 19:49:14 testmac kernel: e1000_get_speed_and_duplex
Feb 9 19:49:14 testmac kernel: 1000 Mbs,
Feb 9 19:49:14 testmac kernel: Full Duplex^M
Feb 9 19:49:14 testmac kernel:
Feb 9 19:49:14 testmac kernel: e1000_force_mac_fc
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_get_speed_and_duplex
Feb 9 19:49:14 testmac kernel: 1000 Mbs,
Feb 9 19:49:14 testmac kernel: Full Duplex^M
Feb 9 19:49:14 testmac kernel:
Feb 9 19:49:14 testmac kernel: e1000: eth1 NIC Link is Up 1000 Mbps Full Duplex
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:14 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:14 testmac kernel: e1000_update_adaptive
Feb 9 19:49:16 testmac kernel: e1000_phy_get_info
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:16 testmac kernel: e1000_phy_m88_get_info
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:16 testmac kernel: e1000_check_polarity
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg
Feb 9 19:49:16 testmac kernel: e1000_read_phy_reg_ex
Feb 9 19:49:16 testmac kernel: e1000_check_for_link
