Howfunky.com

After doing several late night and weekend cutover and integration projects I realized that much of my network troubleshooting ability is not based on a specific checklist of items (though I am going to build one) but on disciplines that I learned while in engineering school.

Specifically, much of my time is spent gathering known variables, quickly writing up what the problem statement and conditions are and then forming a hypothesis that I can work off of to solve the problem at hand. Often, keeping the scope of the problems small and discrete helps, utilizing the engineering principle of KISS (Keep It Simple Stupid) minimize the impact of feeling overwhelmed by an issue. This is especially true when you are in a stressful situation due to limited time or the inability to rollback a solution - both are to be avoid at all cost but often in the world of consulting they are why you are brought into a project in the first place. I guess that goes with the territory.

There are lots of great resources out there that define the principles of engineering so I won't bother with links to those. I have found that the major process I use is not so different from the design principles you use to design and deploy networks. Cisco has a whole methodoly build around this and those that have suffered through their study materials know the PPDIOO mantra.

I think I prefer the more classic engineering school outline, something like:
Identify and understand the problem
Gather information
Generate several solutions
Choose the best of those solutions (KISS)
Prototype the solution
Deploy the solution
Redesign/Retest
Report on results

Each problem you face can be broken down and solved with this method, the difference between those who do network troubleshooting well and those who do not seems to be in the ability to quickly gather and analysis a situation, pick the best solution and then rapidly deploy and tweek the solution. This often comes with age and experience but I am amazed at how many colleagues I have watched over the years who do not follow any of these principles while troubleshooting problems. I primarily have noticed the lack of the engineering principle trait in those who did not go through formal engineer school or a technical trade. I have found that those with a military background adhere to a similar principle though slightly adjusted to the function the military provides. They seem to function well in solving these sort of technical problems also so clearly there is a reason I meet some many former military in the networking field.

Another funny item I have noticed over the years is the fact that no matter how much planning and scripting you do it is the small things that seem to get you. I am not entirely sure why this is, perhaps they are easily overlooked while planning and deploying. Regardless, to catch them the KISS principle is remarkably useful in ferreting out those small problems.

I by no means claim that my engineering degree makes me a better troubleshooter than some of my peers who do not have one, I have meet some amazing people in this field and clearly some people simply have the native instinct and problem solving skills needed to outperform others.
- Ed

Labels: Cisco, Design, Engineering, Network Troubleshooting

From my previous posts about 64-bit vpn clients there is a good addition to the list of options outside of Cisco for Windows 7 64-bit OS and that is Shrew Soft who just released an update to their client which supports Windows 7. The best part is the fact you can import Cisco PCF files. I've had good success when using it will Cisco ASA and PIX vpn configurations and some mixed results with Cisco VPN 3000 concentrator.
The current version is 2.1.5 but the 2.1.4 release worked on my Windows 7 64-bit client also. I have several clients also using it and they have not had any issues either. Hats off to the folks at Shrew Soft - appreciate having a working 64-bit IPsec client that is easy to use for Windows 7.
- Ed

Labels: Cisco, ipsec, Shrew Soft, VPN

I work for a Cisco partner so I get lots of updates on product releases, roadmaps and all the business "justifications" of why they do what they do and how to explain that to customers. I also happen to be a Microsoft MVP and I have switched to using Windows 7 64-bit on a fulltime basis (though I can still boot into my Ubuntu install too if needed.) As someone who uses VPN alot, primarily to do remote support for clients it is incredibly frustrating to NOT have a Cisco supported 64-bit IPSec client for Windows Vista or 7.

The reason I bring this up is Cisco just sent out an announcement titled "Cisco VPN Client v5.0.6 (Windows 7 32-bit support) is now available!" - are you kidding me? That is it?

Cisco, you are hurting your install base of clients, you are NOT doing what is right for them. The statement at the end of the announcement says it all. " 64-bit support is under consideration, but is not yet EC'ed for an upcoming release. This support is available in the Cisco AnyConnect VPN Client today (SSL/DTLS). " Translation - we want you to by a new Cisco solution that works with AnyConnect, if you don't have it we will force you to migrate by not developing a 64-bit IPSec client or integrating that functionality into the AnyConnect client that does support 64-bit.

I do not understand this thinking. What about all the Routers, PIX and VPN Concentrator that are deployed that will not get replaced that do NOT support AnyConnect. Now clients are going to purchase new machines with Windows 7 64-bit and have NO capabilities to VPN back into their network with a Cisco solution (at least not without buying a new Cisco product.) The first thing to pop into my head would be is there a free VPN alternative so I don't have to do this upgrade? Alternately, there are other 3rd party IPSec clients that are supported on Windows 7 like NCP and TheGreenBow but honestly, why should an existing Cisco client have to pay for a new software client when they had one that was working!

I honestly have had more pushback about this one item in meetings than anything else lately. Cisco has corrected the cost difference between IPSec and SSL VPN for the ASA - it is time to correct this also.
- Ed

Labels: 64-bit, Cisco, Microsoft Windows 7, VPN

I will be presenting at the next www.PacITPros.org meeting on Microsoft's Better Together story for Windows Server 2008 R2 and Windows 7. I am going to specifically focus on a new feature in the product(s) called BranchCache. BranchCache is a very interesting approach on Microsoft's part to leverage a client/server OS to perform file caching and pre-positioning functions that traditionally has been approached in the network via appliance solutions.

While BranchCache does not address WAN acceleration and optimization, which many of the network appliance solutions target, it does take care of the file caching portion. For many smaller businesses that is the bulk of traffic in either their VPN or WAN networks. The fact that this is an included feature of the OS just shows how (IMHO) Microsoft is pushing more and more services into software and eliminating the need of special network devices.

I think this just demonstrates the fundamental difference in a company like Cisco who is pushing more services and functions into the network and "cloud" compared to a company like Microsoft who is enabling their operating system to perform and optimize on its own. To be honest, I have no idea who is going to win this one.

I think the challenge will be for Microsoft to play better in an inhomogeneous OS environment, it might mean giving away some of its designs and models and perhaps some code to erode away at positions that clearly favor intelligence in the network. Microsoft is all about software and Cisco is all about network - this I believe is the long term battle that most IT professionals are overlooking today.
- Ed

Labels: Cisco, Microsoft BranchCache

Cisco annual conference is in San Francisco this week. I'll be darting over to attend a Partner event Tuesday evening. In the past I have attended Cisco Networkers but my schedule hasn't allowed me much time to attend.
I might try to make it in to see the expo floor later this week.
On a related note, Cisco announced their newest certification at Live! - Cisco Certified Architect - requires a review board and is above CCIE/CCDE level. Yet another level of BS for the industry IMHO. I am wonder why anyone will bother going to college anymore to earn an Engineering degree at all! LOL
Anyway, check out the announcement here.
- Ed

Labels: Cisco, Live, Networkers

For those that are keeping track there was an interim release on Dec 5th for asa804-16-k8.bin plus as I noted before there is now an ASDM release of 6.1.5.51 or asdm-61551.bin which is compatible with Jave 6.10 or 6.11. There is one issue I have found with the newer ASA code, it does not appear to honor the global translation timeout settings. So if you have long flow sessions (big single tcp backup sessions that stay open forever for instance) then you can have some serious issues.

Also, is anyone else completely baffled as to why the ASA is requiring a separate mobile AnyConnect license, even if you already paid for SSLVPN licenses for the standard AnyConnect on the ASA? Seems like double dipping to me. At a minimum a free license extension should be offered to clients to get some mobile licenses based on the number of existing AnyConnect licenses you already own. Perhaps 2 mobile for every 10 SSLVPN? Cisco, you are just getting greedy on that one.

Happy New Year everyone.
- Ed

Labels: Cisco

Cisco has posted an ASDM update 6.1(5)51 that is compatible with Java 1.6.0_10 (6u10). This takes care of the problem of running the newest Java but having to keep old versions around to support the older ASDM releases that required 6u7.
- Ed

Labels: ASA, ASDM, Cisco

OK, it is very clear now that the UC plan for Cisco is to move all services to Web 2.0 and give customers the choice of either running the solution in house, in the cloud or a combo.
That being said, I don't think it is all baked out yet (they have purchased Jabber but where that is going to fit in with WebEx ? MeetingPlace I am unclear) but I did get the message loud and clear on one thing... the Apple iPhone will be THE platform of future development. I think the iPhone and Windows Mobile will get a lot of attention. I am not very clear how much attention BlackBerry will get, which is odd given its install base. I don't know if Cisco thinks there isn't enough Web 2.0 support on the RIM side or what but I sure didn't get the impression that they were going to make the next "wow" application on the RIM handhelds.
As soon as I hear a clearer roadmap I will post it up. Heck, if anyone knows for sure and has links to back it up please tell me! - Ed

Labels: Cisco

Cisco just finished up their UC Partner VT in San Jose and Web 2.0 services are a huge part of what they are up to with the UC product family. It also seems that Cisco is turning into an everything but Microsoft sort of play. All the product families are on Linux, heavy development on Apple integration and support within the whole Cisco product families plus moving to more "open" standards in regards to directory structures. Its been in the works for awhile but it pretty much officially here now. Cisco is head to head with Microsoft in the UC space and wants to build a large ecosystem around their product families without any MS products in the picture. Never mind the partner part of the "partner / compete" motto.
It also seems with the Web 2.0 push that development on the iPhone as the mobile platform of choice is Cisco's game plan. With the release of the Cisco VPN client on the iPhone plus the fact Apple licensed ActiveSync from Microsoft it really does seem that you will see Cisco use it as the mobile interface to their product family. They are having Cisco employees switching over from the Nokia dual mode handsets to the 3G iPhone... that should tell you something. I wonder what will happen to Blackberry in all this. I might have to move to the iPhone just to start showing off some of the new offering that Cisco will have in the UC space.
- Ed

Labels: Apple, Cisco, Web 2.0

OK, for those of you who play with the Cisco ASA product you might have heard of DAP (Dynamic Access Policies). DAP is used to build policy rules on the fly to provide a customer user experience for VPN sessions (SSLVPN and Clientless or webportal VPN in particular) and is something that has been needed for awhile to compete with the Juniper Neoteris product. DAP has some issues with configuration and setup that can be a challenge, the primary challenge being the Microsoft AD integration.
It turns out that trying to figure out the Login DN parameters can be difficult and also the format for the LDAP attributes. I recommend using LDP to help you figure out the LDAP attributes you can match on and also as a useful tool to walk the LDAP structure of AD. The other missing information is that the testing tool does NOT test against the LDAP authentication server to see if the parameters you are providing actually exist. All it does is TRUST what you are providing as the if that was supplied back from the LDAP server and uses that to test your DAP policy. So you can happily test away thinking your DAP policy will work when it will fail because you are using the wrong LDAP attribut to match in the first place! Very frustrating.
Key commands to know:
debug dap trace
debug ldap 255

Also, for some reason the ASDM DAP testing tool puts commands in the ASA that are cumilative and you have to remove them via the command line. So if you do use the DAP testing tool remember to go in and remove the old parameters you gave it. Otherwise you will have a list a mile long and all of them will be getting checked even though you might only have one or two in the ASDM GUI window.
Oh, and make sure you are running 8.0.3.12, that fixes a SSH issue on the platform that is pretty important.
- Ed

Labels: Cisco