My new Oracle-provided laptop arrived around mid-week and I realized that, at least for now, I’ll have to revert back to using the Windows-based laptop and (hopefully temporarily) put my MacBook Pro on the shelf. Actually, my wife is very excited since she’ll get the MBP to use now and we’ll do the usual “trickle down” to the kids so that the oldest computer in the “fleet” will get ditched.

I tried “upgrading” to a new DSL line (from cable modem) last week too, but that appears to have failed as the DSL modem drops my connection on a regular basis for a few seconds. That’s just long enough to break the VPN connection and make it appear that I’m bouncing off and on instant messaging every hour or so. Annoying.

I was excited to finally get on to some real technical work late last week and got to login to my first Exadata storage server. The chores were to re-image 4 servers and apply the latest patches to them. The real fun is starting now!

Of course, the most important part of any job (in my opinion) is people. I have had several virtual meetings with my peers and manager so far and, as expected, they’re all superb. I’m looking forward to working more with the team and others in our companion teams as well as the days roll ahead.

So, this new opportunity came along a little while ago and I thought it sounded just perfect for me. Many of you that know me will recall that I’ve had trouble finding the right company that fits with all aspects of my personality, goals, and philosophy which has led me to “try” a few of them in the past several years. I don’t regret the choices I’ve made and I’ve learned an awful lot from each of my employers. Most importantly, I’ve created new relationships at each of my past companies that I still maintain today.

In looking at all the past experiences, I’ve concluded that I am ready for a change in direction. Things I enjoy:
Oracle products
People (customers and Oracle employees) that work with Oracle software
RAC
High Availability
Large, complex environments
Servers, OS, and storage infrastructures
Working with lots of new people, especially creating, managing, and growing new relationships
Presenting my knowledge to others, both one-on-one and to groups (and conference events that surround such gatherings)
A little bit of travel (which helps support my love of…)
Scuba diving

So, when an opportunity came along to get paid to do something that combines almost all of the things I enjoy (except scuba), I couldn’t pass it up. Starting on May 18, 2009, I will be the newest member of the new group at Oracle that’s known as the “X-Team”. This team is responsible for working with customers and prospective customers of the Oracle Exadata and HP Oracle Database Machine products to help them adopt these new technologies. The group is a part of the Maximum Availability Architecture group at Oracle that authors many of the best practices whitepapers and presentations you have likely seen online. For those at Oracle that know what this means, the group is a part of the Server Technology development organization under Juan Loaiza.

For those of you that have been my past consulting customers, first of all, Thank You. I’m no longer consulting and while I won’t be able to provide an “Oracle-sanctioned” recommendation to other consultants that may be able to help, I do have a large network of friends and one of them can likely help you. Please never hesitate to keep in touch!

On a logistical note, I’m not moving and will hopefully continue to be involved with local events in Chicagoland. However, I will be traveling part of the time to visit customers and other Oracle facilities, so keep an eye on my twitter feed, Britekite location, and Tripit plans and let me know if there’s a chance to have a meeting IRL.

Finally, this decision to join Oracle means that I’ll be sacrificing several things. First and probably most near and dear to me is the RAC SIG. In September 2008, I took over as the RAC SIG President. The RAC SIG is as strong as ever and there are a good group of volunteers involved in leading the group as it continues to grow and evolve. I’ll always be a member of the RAC SIG and will continue to watch it closely and volunteer when and where I can. The RAC SIG is associated with the IOUG, the Independent Oracle User Group, and Oracle employees shouldn’t be too involved in “independent” groups. So, this year, the RAC SIG will once again elect a new president. I will remain president until Oracle Open World in October 2009 in order to provide continuity to the group’s leadership and ensure a smooth transition. You can nominate yourself for a RAC SIG office soon via our website nomination form (nominations will be open soon and stay open until July 31, 2009).

I’m also going to relinquish my appointment as an Oracle ACE Director. While I think I’ll still be considered an Oracle Employee ACE, I’ll remember fondly the fame that Oracle Technology Network affords the Oracle ACE program and the individuals that are given the honor. Thanks to Justin, Vikki, Lillian, Todd, and the others at Oracle for allowing me to be a part of that program. I’ll certainly miss the perks!

That’s about it for now, I’m off to the new job and will once again begin learning. Luckily, I’m apparently the only person named Dan Norris at Oracle (last someone checked for me), so you can contact me at dan.norris@oracle.com in a couple weeks.

Here are some choice excerpts from the transcript that I find telling about the future of Exadata:

Larry Ellison:

“So, that’s looking back. Now looking forward, I think the most exciting product we’ve had in many, many years is our Exadata Database Server.”

“Exadata is 100% innovation on top of our very large and very strong database business. And the early results have been remarkable. Charles Phillips will go into a lot of detail but I’ll just throw a couple of numbers out there.

One of our customers, and Charles will describe this customer, one of our customers saw a 28x performance improvement over an existing Oracle database. Another customer saw a monthly aggregation drop from 4.5 hours just to 3 minutes.

When compared to Teradata, a competitive database machine that’s been in the market for a very, very long time, another customer saw that we were 6x faster than their existing Teradata application, when using Exadata versus Teradata.

Another customer saw a batch process fall from 8 hours to 30 minutes. Charles will go into more detail on all this, he will repeat those numbers, because I think they’re worth mentioning twice.”

Charles Phillips:

“So now just a few comments by area. On databases, Larry mentioned, we’re very excited about how the HP Oracle database machine is performing. The increases have just been stunning and so we are getting great feedback from our customers and the pipeline is the largest build I’ve ever seen in terms of a new product.

And as he mentioned, the numbers are just stunning. The major European retailer who reduced the batch processing time from 8 hours to 30 minutes did not believe the process had completed. We had to convince him that’s actually how it’s done.

And so, as Larry mentioned, this is the reminder that this is an internally developed technology in the midst of all the discussion of acquisitions. People forget that we’re actually spending $3.0 billion a year on research and development and this is why we do it.”

From these snippets, you can see that the top executives at Oracle are excited about Exadata. If you’re a techie (if you’re not, how’d you get to this blog?), you’ll probably already know about Kevin Closson’s popular blog on all things related to Oracle and storage. Kevin is giving a webcast next week on Exadata where we expect he’ll discuss some of the technical workings of the product–deeper than the overview information many of us have heard before. If you’re interested, I strongly encourage you to sign up for the event and attend. There is no better authority on Exadata than Kevin and this is a great opportunity!

As has been the case for several years, the IOUG and ODTUG have graciously selected some of my submissions for presentation at their respective conferences. While ODTUG has announced that they’ve selected their sessions, they haven’t yet posted them or notified many of the speakers.

For the 5 or so people that probably read this blog , here are the sessions where you’ll find me at Collaborate 08 this year:

• To RAC or Not To RAC: What’s Best For HA? (this is a debate session that I’m coordinating), Thursday @ 9:45 AM
• SSL, Load Balancers, Rewrite, Redirect, and More Advanced Configuration (a 2-hour expert session co-presented with Matt Topper), Monday @ 9:15 AM
• Building A RAC Test Environment On VMWare For Free (a 2-hour expert session), Tuesday @ 3:30 PM
• Oracle Adaptive Access Manager: What, Why, How, Wednesday @ 8:30 AM

Of course, I’ll also be coordinating and participating in several sessions for the RAC SIG at the conference, but we haven’t yet set dates or times for them.

At the ODTUG Kaleidoscope conference happening June 15-18 in New Orleans, I’ll be presenting the following sessions (they haven’t scheduled them yet):

• How A RAT Will Save Your Job
• RAC For Beginners: The Basics
• Sunday Fusion Middleware Symposium session: SSL, Load Balancers, Rewrite, Redirect, and More Advanced Configuration (co-presentation with Matt–a repeat of the Collaborate session)

UPDATE: The last session listed above (OAS Advanced Config) is only to be presented at the Sunday Fusion Middleware Symposium instead of a regular conference slot. If you really want the full version, come to Collaborate!

If you submitted a session to ODTUG and haven’t heard back, check here for status.

Still not enough? I’m also presenting at the Northern California Oracle Users Group conference on February 19th at Oracle’s Conference Center on the HQ Campus. At that event, you can see me in the front talking on these topics:

• Oracle Adaptive Access Manager: What, Why, How, at 2:30 PM
• High Availability Options for Oracle Database, at 4 PM

It’ll be a busy spring and summer. I’m not sure I really understood how busy until I wrote it all down in one place…yikes.

Then things might calm down, but not for long since Oracle OpenWorld is coming early this year (September 21-25)!

I thought Jake’s 8 things about himself were pretty interesting. You know how when you talk to someone on the phone and immediately start creating a mental image of them? Later, you finally get to have a meeting in person and find out you were waaaayyyy off? As anyone that has met me or seen me knows, I’m not a particularly vain person. If it weren’t for the fact that I try to make a living and others generally judge me based on my appearance, I’d wear cutoffs swimming trunks (I am a diver as you’ll learn in a moment) and t-shirts with funny, almost inappropriate sayings on them all the time. I once asked my grandfather how old I had to be in order to stop caring whether my fly is open (note that most people won’t tell an “old person” when their fly is open). He said 62, not sure why, but he’s the authority on such things. If you ever saw him, you’d know.

Anyway, to delight your imaginations and give you many more reasons to change your mental image of my persona, here are 8 things about me.

1. My entire family attended all of my high school functions. My dad was the high school principal and I had to leave my date at prom, homecoming…pretty much all functions for about 15 minutes to get a family photo taken. In between functions, just trying to get everyone to forget that you’re the principal’s kid was a full time job.
2. I can operate pretty much every piece of machinery found on a grain farm from planter to cultivator to combine. From the time I was 12 until college, I worked all summer and weekends in the fall and spring as the sole farmhand for 2500 acres of corn and soybeans. Great tan, kept in shape, made some cash, and learned a lot about mechanical things too. Sometimes, I still yearn to work a weekend or two in the fall when harvest starts. There’s a funny story about getting followed by an unmarked sheriff’s deputy car while driving the pickup back to the farm (I was 13 at the time and not driving legally). Turns out, my boss’ brother was the sheriff who was just stopping by for a visit…I almost soiled myself.
3. I was a crew trainer at McDonald’s when I was 16. Only those that have experienced it can attest to the permanent smell of recon onions and pickles that sticks around for 2-3 days after you work a shift on the grill. I made the biscuits (at 4:30am) back when McDonald’s used to make fresh biscuits in every location. They’re shipped in frozen now–fresh was better. Also note that if you ever saw a McRib before they covered it in that sauce, you’d never eat one again.
4. I’m a scuba diver. After 6 years of wanting to learn, I finally got around to it (delay due to children) this year. I am currently a PADI Advanced Open Water Diver and am working on my Rescue Diver certification as soon as it thaws out in Illinois. I keep my scuba diving page here up-to-date…mostly.
5. One of my favorite times in life was working as an Assistant Stage Manager at Illinois State University. As part of the student stage hands crew, I got to work on setting up and tearing down roadshows that came through Bloomington-Normal, Illinois on their tours. I worked on shows for Travis Tritt, Marty Stuart, Megadeth, Extreme, Stone Temple Pilots, Toad the Wet Sprocket, the launch of the Smashing Pumpkins world tour, Buddy Guy (awesome), Harry Belafonte, Bill Cosby, Gallagher, and a bunch of others I can’t remember. I used to mix sound for some local bands too. Unforuntely, I was only there two years, but getting paid to hang out back stage is pretty cool, even if only for a little while. (I was at Illinois State for 2 years before transferring to UIUC where I graduated 2.5 years later.)
6. My brother is an orthopedic surgeon. Those are the guys that bring drills, screws, pins, plates, and fake joints into an operating room. Weird stuff. I’ve learned that when he wants to show me pictures of the place he went last weekend, I have to be prepared for the accidental picture of some dude’s broken femur to be “accidentally” stuck in the middle of the stack. I think he just does it to test my constitution–he secretly remembers all those times I beat him up when we were younger. Mom always told me that someday he’d be bigger than me…(and he is that too, but has kindly chosen photographic torture instead). Regardless of our past, I’m very proud of him.
7. My wife is a (great, goes without saying really) writer. I suspect like all great writers, she writes many things that she immediately throws away claiming that it isn’t worthy of reading. If there is an opposite to my work, creative fiction writing is probably about as close as it can get. She doesn’t get much time to work on writing due to taking care of Kids 2.0 (including updating their website) and House 3.0 (this is our 3rd ownership experience). She’s wonderful and her strengths complement all my weaknesses perfectly. Plus, she’s one helluva cook!
8. I like to travel. Well, not constantly, but I’d rather buy a trip to some interesting place than to buy a “thing” for myself. I enjoy the occasional travel for work. Unfortunately, it usually comes in batches (gone for a month, home for a month) which is tiring and stressful for my family, but I generally enjoy going to new places. With the New Web, I am especially enjoying my ability to find people in the places I go and (at least try to) connect with them. Since I’ve recently added diving to my hobby list (there’s one on the list, diving), and I live in Chicagoland, travel will become more important as the diving in and around where I live is only truly fun for a few months in the summer. After that, it’s tolerable, but I digress…

So, there you have them, 8 things about me. I know, some are really about other people, but they are people that define me, so that’s why they’re on the list. Now for the fun part–8 people to tag for the next round.

I hereby officially tag the following individuals: Jeremy Schneider, Lewis Cunningham, Billy Cripe, Chris Muir, Charles Schultz, Mogens Nørgaard (a.k.a. Moans Nogood), Mark Rittman, and Alex Gorbachev. I’ll be emailing or tweeting you shortly to let you know you’ve been tagged!

Update: I was only half joking about the RAC support, but now I’ve had time to read the FAQ and found this: “Oracle Real Application Clusters testing is in progress and it will be supported with Oracle VM in 2008 (calendar year).” …Excellent!

I have a lot of plans to attend some of the good RAC and Identity Management sessions at OOW. I won’t bore you with my personal pick list, but if you’re looking for me (because you want to hear a good joke or I owe you money or some such thing), you’ll definitely find me at these sessions since I’m involved in them to some extent:

• Sunday, 11-Nov
  • 10 am: High Availability Options for Oracle Database, Moscone West 2003
  • 1 pm: Is that really you? Prove it!, Moscone West 2005
  • 2:30 pm: 11g Database Beta Testing Panel, Moscone West 2004
  • 4 pm: RAC SIG Birds of a Feather session, Moscone West 2002
• Monday, 12-Nov
  • 10 am: Piocon Booth, Moscone South
  • 12:30 pm: RAC SIG Customer Panel, Moscone South 310
  • 4:45 pm: RAC For Beginners: The Basics, Hilton Continental Ballroom 6
  • 7:30 pm: OTN Night, Westin St. Francis Hotel
• Tuesday, 13-Nov
  • 10 am: Piocon Booth, Moscone South
  • 12:15 pm: RAC SIG Expert Panel
  • 3:15 pm: Demystifying Fusion Middleware, Hilton Yosemite Room C
• Wednesday, 14-Nov
  • 9 am: Piocon Booth, Moscone South
  • 11:15 am: other technical sessions
  • 3 pm: Piocon Booth, Moscone South
  • 8 pm: PARTY @ Cow Palace
• Thursday, 15-Nov
  • 10 am: Piocon Booth, Moscone South
  • 11:30 am: other technical sessions
  • 5 pm: It’s a wrap reception, Howard Street

Should be a fun week. Please stop by and meet me at one of these places or anywhere else you happen to see me. Now, back to preparing the presentations for my sessions . See you in SF! Travel safe.

I’ll be joining several other session proposers at the Unconference event on Monday to reserve a time and place to hold this session. Once I have a slot reserved, I’ll update the page on the wiki with time and location as well as posting it here.

Please post something there or contact me directly if you’re interested in reviewing some of the common settings, discussing ways to work around limitations in the application server, applications, and other tips and tricks. I plan for this session to be highly interactive, but I firmly believe that everyone will leave with at least one new idea, technique, or tip in their brain.

See you there–only 3 more weeks!

We increased many timeouts for OPMN as we believed that OPMN was just incorrectly “seeing” the containers as down and restarting them. OPMN restarts them by shutting them down first and then starting them.

We filed cases with Oracle support to no avail–they didn’t come up with any useful suggestions in a week or more. They were trying, but didn’t come up with the solution.

The system administrator developed a theory based on what he believed was a pattern. Every time he did a deployment, he would notice a crash of all the non-Oracle default containers. That is, the home and OC4J_WebCenter containers didn’t crash.

The deployment process that he followed resulted in him connecting to the server using remote desktop. His remote desktop client was configured with the /console option which was required by some other servers he managed, more about that later.

Once he was able to demonstrate that he could make the containers crash each time he logged off, we started testing variations using the system console, the remote desktop client with and without the /console option and found a pattern. The remote desktop client without the /console option did not cause a crash, but all other combinations did. Through all of this, the home and OC4J_WebCenter containers remained up and running.

Bottom line: Read Metalink Note 245609.1 which documents the apparently, well-known fact that logging out from the Windows console causes JVM termination. The very simple fix is to start the containers with the “-Xrs” option which tells the JVM to ignore certain signals from the OS.

The really terrible thing about all this is that Oracle puts the -Xrs option on the containers deployed during the installation, but the OEM tool doesn’t add them to the container startup parameters for the custom containers. Easy to fix, even easy to find once you know what to look for.

This begs two questions:

1. Why doesn’t Oracle add -Xrs to the startup options for the containers created after the initial installation? That would have avoided all the problems and there’s apparently no negative side effect–at least not that we’ve seen.
2. How could an SR analyst not find this Metalink note and refer us to the simple solution? Granted, we didn’t find it easily in our searches either, but eventually it was one of us that found the article and solution. Now that we know the fix, a simple search for -Xrs on Metalink gets plenty of hits. As they say, hindsight is 20/20.

Hopefully, this information will help some of you that are lucky enough to work on OC4J deployments on Windows.