====== Automatisierung mit event-action und Python ====== An diesem Artikel wird noch gearbeitet. -benny Die Funktion "event-action" erlaubt es im AOS Release 7.3.4.R01/R02 automatisch auf Vorkommnisse im Netzwerk zu reagieren. Dies kann ein einfacher Port sein der keinen Link mehr hat "linkDown" oder auch ein komplizierterer Vorgang wie der Ausstieg eines einzelnen Ports aus einer Link-Aggregation "lnkaggPortLeave" sein. (Der Unterschied ist dass im Fall des "linkDown" der Port physikalisch down ist, während im Fall des "lnkaggPortLeave" lediglich das Control-Protocol (LACP) den Ausstieg signalisiert. (Dies könnte u.a. auftreten wenn die Verbindung über eine DWDM-Verbindung geführt wird, die den Port noch "up" hält aber aus anderen Gründen keine durchgängige Verbindung mehr schaltet.) ====== Funktionsweise der "event-action" Aktion ====== ===== 10G/1G Link-Aggregation - Backup-Strecke aktivieren ===== Eine Link-Aggregation mit unterschiedlichen Geschwindigkeiten ist nicht offiziell unterstützt. Dies macht aus Gründen von "Hashing" über die zur Verfügung stehenden Verbindungen auch keinen Sinn (Hashing bezieht nicht die Geschwindigkeite mit in die Entscheidung ein). Angenommen wir haben eine Link-Aggregation auf den Ports 1/1/1a (10G) und 1/1/1b (1G), so kann folgendes Event-Action Skript für uns die 1G Strecke immer dann aktiv schalten wenn die 10G Verbindung aus Sicht des Control-Protocol (LACP) nicht verfügbar ist. Wenn die Strecke zurückkommt, wird der 1G Port wieder ausgeschaltet. Der Vorgang hat eine Wiederherstellungszeit von ca. 2 Sekunden. #!/bin/python3 import sys import os import getopt import json # Based on work by Patricio Martelo # Adaption for Link-Aggregation management by Benny # TODO: Delete, just for tests #print(sys.argv) # Load the data that is being sent to us # -t holds the traptype # -d holds the trapdata try: opts, args = getopt.getopt(sys.argv[1:], "t:d:") except getopt.GetoptError as err: print(error) print("%s" % sys.argv) sys.exit(2) traptype = "(none)" trapdata = "{}" # Go through the data in opts and allocate it properly # traptype gets the value from -t # trapdata gets the value from -d for o, a in opts: if o == "-t": traptype = a elif o == "-d": trapdata = a # Load the data from json format into Python dictionary trapdata = json.loads(trapdata); if traptype == 'lnkaggAggDown': if trapdata['traplnkaggPortIfIndex'] == 1001: print('Detected 10G port leaving the aggregate, bringing up backup 1G.\n') os.system("interfaces 1/1/1b admin-state enable") elif traptype == 'lnkaggAggUp': if trapdata['traplnkaggPortIfIndex'] == 1002: print('Restored the operation, running on 1G backup.\n') elif traptype == 'lnkaggPortLeave': if trapdata['traplnkaggPortIfIndex'] == 1001: print('Detected 10G port leaving the aggregate, bringing up backup 1G.\n') os.system("interfaces 1/1/1b admin-state enable") elif traptype == 'lnkaggPortJoin': if trapdata['traplnkaggPortIfIndex'] == 1001: print('Detected 10G port joining the aggregate, bringing down backup 1G.\n') os.system("interfaces 1/1/1b admin-state disable") Ausgabe auf der Console: Detected 10G port leaving the aggregate, bringing up backup 1G. Restored the operation, running on 1G backup. Detected 10G port joining the aggregate, bringing down backup 1G. OS6900-> OS6900-> OS6900-> show linkagg port Chassis/Slot/Port Aggregate SNMP Id Status Agg Oper Link Prim -------------------+----------+--------+----------+----+-----+-----+---- 1/1/1A Dynamic 1001 ATTACHED 1 UP UP YES 1/1/1B Dynamic 1002 CONFIGURED NONE DOWN DOWN UNK OS6900-> Auswirkung auf den Betrieb: 64 bytes from 192.168.2.2: icmp_seq=39 ttl=64 time=0.971 ms 64 bytes from 192.168.2.2: icmp_seq=40 ttl=64 time=0.888 ms ping: sendmsg: Network is unreachable ping: sendmsg: Network is unreachable 64 bytes from 192.168.2.2: icmp_seq=44 ttl=64 time=12.3 ms 64 bytes from 192.168.2.2: icmp_seq=45 ttl=64 time=0.855 ms 64 bytes from 192.168.2.2: icmp_seq=46 ttl=64 time=0.880 ms 64 bytes from 192.168.2.2: icmp_seq=47 ttl=64 time=0.871 ms 64 bytes from 192.168.2.2: icmp_seq=48 ttl=64 time=0.876 ms 64 bytes from 192.168.2.2: icmp_seq=49 ttl=64 time=19.6 ms 64 bytes from 192.168.2.2: icmp_seq=50 ttl=64 time=1.36 ms Beim Zurückschalten wurden keine Pakete verloren (1G Verbindung wird deaktiviert). ====== Tipps ====== ===== Verwendung von Python DeBugger (pdb) im Rahmen von "event-action" nicht möglich ===== Obwohl die Verwendung von pdb generell möglich ist, wird dies im Rahmen von event-action Skripten nicht unterstützt. Dies hängt u.a. mit dem automatischen Ablauf und der maximalen Laufzeit von 60 Sekunden für diese Skripte zusammen. Der Versuch erzeugt die Exception "BdbQuit". #!/bin/python3 import pdb pdb.set_trace() ====== TODO ====== * sys.argv * Erläuterung wie die Ports hochzählen => Q32 * trap-absortion ausmachen! * Traps für linkDown / linkUp einschalten OS6900-> interfaces 1/1/2 link-trap enable OS6900-> interfaces 1/1/3 li link-monitoring link-trap OS6900-> interfaces 1/1/3 link-trap enable OS6900-> show event-action Script Time Limit (seconds): 60 Type Name Script (/flash/python/...) ------+---------------------------------------+---------------------------------- trap linkUp link_agg_mgr.py trap linkDown link_agg_mgr.py trap lnkaggAggUp link_agg_mgr.py trap lnkaggAggDown link_agg_mgr.py trap lnkaggPortJoin link_agg_mgr.py trap lnkaggPortLeave link_agg_mgr.py OS6900-> Sun Mar 23 04:49:18 : ChassisSupervisor MipMgr info message: +++ Trap Sent from CCM: Entity Config Change Trap ['/flash/python/link_agg_mgr.py', '-t', 'linkUp', '-d', '{"sysUpTime":942008,"ifIndex":1005,"ifAdminStatus":1,"ifOperStatus":1}'] ['/flash/python/link_agg_mgr.py', '-t', 'linkUp', '-d', '{"sysUpTime":942009,"ifIndex":1009,"ifAdminStatus":1,"ifOperStatus":1}'] ['/flash/python/link_agg_mgr.py', '-t', 'linkDown', '-d', '{"sysUpTime":942637,"ifIndex":1005,"ifAdminStatus":1,"ifOperStatus":2}'] ['/flash/python/link_agg_mgr.py', '-t', 'linkDown', '-d', '{"sysUpTime":942639,"ifIndex":1009,"ifAdminStatus":1,"ifOperStatus":2}'] Sun Mar 23 04:49:26 : ChassisSupervisor MipMgr info message: +++ Trap Sent from CCM: Entity Config Change Trap OS6900-> ['/flash/python/link_agg_mgr.py', '-t', 'lnkaggAggDown', '-d', '{"sysUpTime":919117,"traplnkaggAggId":1,"traplnkaggPortIfIndex":1001}'] OS6900-> interfaces 1/1/1a admin-state enable OS6900-> ['/flash/python/link_agg_mgr.py', '-t', 'lnkaggAggUp', '-d', '{"sysUpTime":921356,"traplnkaggAggId":1,"traplnkaggPortIfIndex":1001}'] OS6900-> ['/flash/python/link_agg_mgr.py', '-t', 'lnkaggPortLeave', '-d', '{"sysUpTime":907812,"traplnkaggAggId":1,"traplnkaggPortIfIndex":1002}'] OS6900-> interfaces 1/1/1b admin-state enable OS6900-> ['/flash/python/link_agg_mgr.py', '-t', 'lnkaggPortJoin', '-d', '{"sysUpTime":875582,"traplnkaggAggId":1,"traplnkaggPortIfIndex":1002}'] OS6900-> OS6900-> interfaces 1/1/1a admin-state disable OS6900-> ['/flash/python/link_agg_mgr.py', '-t', 'lnkaggPortLeave', '-d', '{"sysUpTime":901525,"traplnkaggAggId":1,"traplnkaggPortIfIndex":1001}'] OS6900-> interfaces 1/1/1a admin-state enable OS6900-> ['/flash/python/link_agg_mgr.py', '-t', 'lnkaggPortJoin', '-d', '{"sysUpTime":902581,"traplnkaggAggId":1,"traplnkaggPortIfIndex":1001}'] ====== SNMP-Trap Übersicht ====== OS6900-> show snmp-trap config Absorption service : disabled Traps to WebView : enabled id trap name family absorption --+------------------------------------+---------------+------------ 0 coldStart chassis 15 seconds 1 warmStart chassis 15 seconds 2 linkDown interface 15 seconds 3 linkUp interface 15 seconds 4 authenticationFailure snmp 15 seconds 5 entConfigChange module 15 seconds 6 policyEventNotification qos 15 seconds 7 chassisTrapsStr chassis 15 seconds 8 chassisTrapsAlert chassis 15 seconds 9 chassisTrapsStateChange chassis 15 seconds 10 chassisTrapsMacOverlap module 15 seconds 11 vrrpTrapNewMaster vrrp 15 seconds 12 vrrpTrapAuthFailure vrrp 15 seconds 13 healthMonModuleTrap health 15 seconds 14 healthMonPortTrap health 15 seconds 15 healthMonCmmTrap health 15 seconds 16 bgpEstablished bgp 15 seconds 17 bgpBackwardTransition bgp 15 seconds 18 esmDrvTrapDropsLink interface 15 seconds 19 portViolationTrap interface 15 seconds 20 dvmrpNeighborLoss ipmr 15 seconds 21 dvmrpNeighborNotPruning ipmr 15 seconds 22 risingAlarm rmon 15 seconds 23 fallingAlarm rmon 15 seconds 24 stpNewRoot stp 15 seconds 25 stpRootPortChange stp 15 seconds 26 mirrorConfigError pmm 15 seconds 27 mirrorUnlikeNi pmm 15 seconds 28 slbTrapOperStatus loadbalancing 15 seconds 29 sessionAuthenticationTrap session 15 seconds 30 trapAbsorptionTrap none no 31 alaDoSTrap ip 15 seconds 32 ospfNbrStateChange ospf 15 seconds 33 ospfVirtNbrStateChange ospf 15 seconds 34 lnkaggAggUp linkaggregation 15 seconds 35 lnkaggAggDown linkaggregation 15 seconds 36 lnkaggPortJoin linkaggregation 15 seconds 37 lnkaggPortLeave linkaggregation 15 seconds 38 lnkaggPortRemove linkaggregation 15 seconds 39 monitorFileWritten pmm 15 seconds 40 alaVrrp3TrapProtoError vrrp 15 seconds 41 alaVrrp3TrapNewMaster vrrp 15 seconds 42 chassisTrapsPossibleDuplicateMac chassis 15 seconds 43 lldpRemTablesChange aip 15 seconds 44 pimNeighborLoss ipmr 15 seconds 45 pimInvalidRegister ipmr 15 seconds 46 pimInvalidJoinPrune ipmr 15 seconds 47 pimRPMappingChange ipmr 15 seconds 48 pimInterfaceElection ipmr 15 seconds 49 pimBsrElectedBSRLostElection ipmr 15 seconds 50 pimBsrCandidateBSRWinElection ipmr 15 seconds 51 lpsViolationTrap bridge 15 seconds 52 lpsPortUpAfterLearningWindowExpiredT bridge 15 seconds 53 lpsLearnTrap bridge 15 seconds 54 gvrpVlanLimitReachedEvent bridge 15 seconds 55 alaNetSecPortTrapAnomaly netsec 15 seconds 56 alaNetSecPortTrapQuarantine netsec 15 seconds 57 ifMauJabberTrap interface 15 seconds 58 udldStateChange interface 15 seconds 59 ndpMaxLimitReached ip 15 seconds 60 ripRouteMaxLimitReached rip 15 seconds 61 ripngRouteMaxLimitReached ripng 15 seconds 62 alaErpRingStateChanged bridge 15 seconds 63 alaErpRingMultipleRpl bridge 15 seconds 64 alaErpRingRemoved bridge 15 seconds 65 ntpMaxAssociation ntp 15 seconds 66 ddmTemperatureThresholdViolated interface 15 seconds 67 ddmVoltageThresholdViolated interface 15 seconds 68 ddmCurrentThresholdViolated interface 15 seconds 69 ddmTxPowerThresholdViolated interface 15 seconds 70 ddmRxPowerThresholdViolated interface 15 seconds 71 webMgtServerErrorTrap webmgt 15 seconds 72 multiChassisIpcVlanUp mcm 15 seconds 73 multiChassisIpcVlanDown mcm 15 seconds 74 multiChassisMisconfigurationFailure mcm 15 seconds 75 multiChassisHelloIntervalConsisFailu mcm 15 seconds 76 multiChassisStpModeConsisFailure mcm 15 seconds 77 multiChassisStpPathCostModeConsisFai mcm 15 seconds 78 multiChassisVflinkStatusConsisFailur mcm 15 seconds 79 multiChassisStpBlockingStatus mcm 15 seconds 80 multiChassisLoopDetected mcm 15 seconds 81 multiChassisHelloTimeout mcm 15 seconds 82 multiChassisVflinkDown mcm 15 seconds 83 multiChassisVFLMemberJoinFailure mcm 15 seconds 84 alaDHLVlanMoveTrap vlan 15 seconds 85 alaDhcpClientAddressAddTrap ip-helper 15 seconds 86 alaDhcpClientAddressExpiryTrap ip-helper 15 seconds 87 alaDhcpClientAddressModifyTrap ip-helper 15 seconds 88 vRtrIsisDatabaseOverload isis 15 seconds 89 vRtrIsisManualAddressDrops isis 15 seconds 90 vRtrIsisCorruptedLSPDetected isis 15 seconds 91 vRtrIsisMaxSeqExceedAttempt isis 15 seconds 92 vRtrIsisIDLenMismatch isis 15 seconds 93 vRtrIsisMaxAreaAddrsMismatch isis 15 seconds 94 vRtrIsisOwnLSPPurge isis 15 seconds 95 vRtrIsisSequenceNumberSkip isis 15 seconds 96 vRtrIsisAutTypeFail isis 15 seconds 97 vRtrIsisAuthFail isis 15 seconds 98 vRtrIsisVersionSkew isis 15 seconds 99 vRtrIsisAreaMismatch isis 15 seconds 100 vRtrIsisRejectedAdjacency isis 15 seconds 101 vRtrIsisLSPTooLargeToPropagate isis 15 seconds 102 vRtrIsisOrigLSPBufSizeMismatch isis 15 seconds 103 vRtrIsisProtoSuppMismatch isis 15 seconds 104 vRtrIsisAdjacencyChange isis 15 seconds 105 vRtrIsisCircIdExhausted isis 15 seconds 106 vRtrIsisAdjRestartStatusChange isis 15 seconds 107 mvrpVlanLimitReachedEvent bridge 15 seconds 108 alaHAVlanClusterPeerMismatch ha-vlan 15 seconds 109 alaHAVlanMCPeerMismatch ha-vlan 15 seconds 110 alaHAVlanDynamicMAC ha-vlan 15 seconds 111 unpMcLagMacIgnored da-unp 15 seconds 112 unpMcLagConfigInconsistency da-unp 15 seconds 113 multiChassisGroupConsisFailure mcm 15 seconds 114 multiChassisTypeConsisFailure mcm 15 seconds 115 alaPimNonBidirHello ipmr 15 seconds 116 dot1agCfmFaultAlarm bridge 15 seconds 117 alaSaaIPIterationCompleteTrap system 15 seconds 118 alaSaaEthIterationCompleteTrap system 15 seconds 119 alaSaaMacIterationCompleteTrap system 15 seconds 120 virtualChassisStatusChange vcm 15 seconds 121 virtualChassisRoleChange vcm 15 seconds 122 virtualChassisVflStatusChange vcm 15 seconds 123 virtualChassisVflMemberPortStatusCh vcm 15 seconds 124 virtualChassisVflMemberPortJoinFail vcm 15 seconds 125 lldpV2RemTablesChange aip 15 seconds 126 vRtrLdpInstanceStateChange mpls 15 seconds 127 evbFailedCdcpTlvTrap evb 15 seconds 128 evbFailedEvbTlvTrap evb 15 seconds 129 evbUnknownVsiManagerTrap evb 15 seconds 130 evbVdpAssocTlvTrap evb 15 seconds 131 evbCdcpLldpExpiredTrap evb 15 seconds 132 evbTlvExpiredTrap evb 15 seconds 133 evbVdpKeepaliveExpiredTrap evb 15 seconds 134 smgrServiceError svcmgr 15 seconds 135 smgrServiceHwError svcmgr 15 seconds 136 smgrSapError svcmgr 15 seconds 137 smgrSapHwError svcmgr 15 seconds 138 smgrSdpError svcmgr 15 seconds 139 smgrSdpHwError svcmgr 15 seconds 140 smgrSdpBindError svcmgr 15 seconds 141 smgrSdpBindHwError svcmgr 15 seconds 142 smgrGeneralError svcmgr 15 seconds 143 smgrStatusChange svcmgr 15 seconds 144 portViolationNotificationTrap interface 15 seconds 145 multiChassisConsisFailureRecovered mcm 15 seconds 146 alaSaaPacketLossTrap system 15 seconds 147 alaSaaJitterThresholdYellowTrap system 15 seconds 148 alaSaaRTTThresholdYellowTrap system 15 seconds 149 alaSaaJitterThresholdRedTrap system 15 seconds 150 alaSaaRTTThresholdRedTrap system 15 seconds 151 chassisTrapsDuplicateMacCleared chassis 15 seconds 152 alaFipsResourceThresholdReached fips 15 seconds 153 virtualChassisUpgradeComplete vcm 15 seconds 154 appFPSignatureMatchTrap appfp 15 seconds 155 virtualChassisVflSpeedTypeChange vcm 15 seconds 156 alaSIPSnoopingACLPreemptedBySOSCall qos 15 seconds 157 alaSIPSnoopingRTCPOverThreshold sip-snooping 15 seconds 158 alaSIPSnoopingRTCPPktsLost qos 15 seconds 159 alaSIPSnoopingSignallingLost qos 15 seconds 160 alaSIPSnoopingCallRecordsFileMoved sip-snooping 15 seconds 161 alaIPv6NeighborLimitExceeded ip 15 seconds 162 alaIPv6NeighborVRFLimitExceeded ip 15 seconds 163 alaIPv6InterfaceNeighborLimitExceed ip 15 seconds 164 alaDyingGaspTrap interface 15 seconds 165 alaDhcpSrvLeaseUtilizationThreshold dhcp-server 15 seconds 166 alaDHCPv6SrvLeaseUtilizationThresho dhcpv6-server 15 seconds 167 smgrServiceStatusChange svcmgr 15 seconds 168 smgrSapStatusChange svcmgr 15 seconds 169 smgrSdpStatusChange svcmgr 15 seconds 170 smgrSdpBindStatusChange svcmgr 15 seconds 171 alaPethPwrSupplyConflictTrap module 15 seconds 172 alaPethPwrSupplyNotSupportedTrap module 15 seconds 173 chasTrapsBPSLessAllocSysPwr chassis 15 seconds 174 chasTrapsBPSStateChange chassis 15 seconds 175 chasTrapsNiBPSFETStateChange chassis 15 seconds 176 alaDhcpBindingDuplicateEntry ip-helper 15 seconds 177 alaVCSPProtectionTrap vcm 15 seconds 178 alaVCSPRecoveryTrap vcm 15 seconds 179 pethPsePortOnOffNotification module 15 seconds 180 pethMainPowerUsageOnNotification module 15 seconds 181 pethMainPowerUsageOffNotification module 15 seconds 182 chasTrapsBPSFwUpgradeAlert chassis 15 seconds 183 alaAppMonAppRecordFileCreated app-mon 15 seconds 184 alaAppMonFlowRecordFileCreated app-mon 15 seconds 185 alaDPIFlowRecordFileCreated dpi 15 seconds 186 alaLbdStateChangeToShutdown lbd 15 seconds 187 alaLbdStateChangeForClearViolationA lbd 15 seconds 188 alaLbdStateChangeForAutoRecovery lbd 15 seconds 189 alaAutoConfigAutoFabricEnableTrap unknown 15 seconds 190 alaVMSnoopingVMLearntAlert vm-snooping 15 seconds 191 alaVMSnoopingVMRemovedAlert vm-snooping 15 seconds 192 alaVMSnoopingReservedHwResourceLimi vm-snooping 15 seconds 193 alaDistArpItfChange ip 15 seconds 194 alaDistArpNiThreshold ip 15 seconds 195 smgrVxlanSdpBindStatusChange svcmgr 15 seconds 196 alaAutoFabricSTPModeChangeAlert fips 15 seconds