Interviewing Oracle DBAs

You may have seen in an earlier article I was trying to drum up interest in a recruitment drive we were having. Well, we whittled down the applicants and recently held the interviews. There was one candidate that stood out quite away above the other hopefuls. I thought it might be of interest to go through some of the interview questions that we used. I know Howard J. Rogers (currently and perhaps permanently off air) has in the past written a fantastic series of pieces on his interview questions, but I feel our experience is worth sharing. Though it seems the lack of good quality Oracle DBAs is a universal theme.

First off, I think it is fair to say the interview was split roughly in two, with the first half covering more attitudinal questions and the second half covering more technical Oracle questions. We certainly place as much weight on finding the correct sort of candidate, one with a good attitude to learning and customer service, as much as what particular level of Oracle skills they have. Also it’s worth pointing out this is for a job in a small company, where there is no strong demarcation of responsibilities, so the DBAs are expected to do a goodly amount of system administration work, and have ownership of configuring the SANs. The candidate we chose I would say displayed excellent answers in both the attitudinal and technical questions. Here are the Oracle questions we asked, I think these are pretty straightforward and gentle questions, there are not even that many of them!

What approach would you take if asked to investigate a query that was running slowly?

A couple of candidates answered this pretty well. I was looking for things like checking the explain plan, and tracing the session. I was not looking for someone to mention the buffer cache hit ratio - though someone did!

What SQL commands will cause a sort to take place?

Unfortunately a few candidates did not get past order by (which all got), some said distinct and group by. The candidate we hired mentioned these and UNION. No one mentioned create index.

How does Oracle implement read-consistency?

Quite a few candidates stumbled at this question. It is such a fundamental part of the Oracle RDBMS that you would hope a candidate would at least have go at answering. A couple of candidates said they were stumped. I was looking for them to mention that a query only sees data committed before the start of the query and uses undo to reconstruct any data if it finds it with a commited time that was after the start of the query. No one mentioned SCN.

Do you know what ORA-01555 is?

This is somewhat related to the previous question. Practically every candidate did not know this. I do not expect a candidate to know every single Oracle error message, I certainly don’t. However 1555 is such a classic error that I would expect good dbas to know what it is. The one candidate who did know that it’s snapshot too old, got the job.

What resources do you use to keep up-to-date with Oracle?

Well I probably was a little disappointed with the responses to this one, only one candidate mentioned a good spread of bloggers, or indeed blogs at all. Everyone mentioned OTN, but no one mentioned actually downloading a new release and trying it out for themselves. Oh and this question lead on to which authors the candidates liked, Tom Kyte was the definite winner with Steven Feuerstein in second place. I was astounded not one candidate mentioned Jonathan Lewis.

Explain the architecture of RAC?

So, I realise this could potentially lead to a wide ranging answer, but most of the candidates had RAC on their CV so you’d expect them to able to explain it at least to a level that they new it was different from single instance Oracle. Minimum requirement was multiple instances accessing the same database datafiles, communicating through a private interconnect, which most met.

Finally, here is one of the non-technical questions we asked:

What kind of working environment do you need to be most effective?

I’ll leave it to the reader to work out what we expected from this one!

Advertisement: Nominet are recruiting

This is a bit of an advertisement for my current employer, Nominet, who are one of the largest internet registries in the world, and run the .uk ccTLD. We are currently looking for an additional DBA. You will be able to get your hands dirty on RAC and Dataguard, we currently use 10gR2 but will be looking to upgrade to 11g some time this year. Nominet actively encourages you to blog about your work and there is plenty of opportunity to become involved with the UKOUG.

You will need a good amount of experience with Oracle but an ability to learn is probably just as important as number of years clocked up. A working knowledge of Unix/Linux would be beneficial as we have to support the full stack from carving up the storage to configuring the applications. The technical department has an excellent can-do attitude and you probably will have too. Nominet is a comfortable working environment and there is also the bonus that you would be able to work alongside some excellent people

I have been involved in several recruitment rounds with Nominet and I have been really astounded at the lack of applicants that we have received. Is there really a dearth of DBAs out there? I got round to concluding it was the location, as the rates of pay on offer are competitive compared to other non-city of London positions. We also seemed to attract large number of applicants with dubious work permits. I had to laugh when I saw a candidate who claimed to be OCP certified and had this stamped onto his cv, but in the text had misspelled Oracle!

I have also been surprised when an interview candidate mentioned the buffer cache hit ratio in response to a question regarding a good way of investigating users complaining about performance problems. They did not get the job. I was amazed, I think partly, due to the advent of blogging. Compared to even a few years ago, there is such a wealth of excellent information out there in the blogosphere, that I think any person claiming to be a DBA who is not keeping up-to-date with current Oracle best practice just has no excuse as it’s never been easier.

So if you or someone you know is interested, then the full details are on the Nominet website. Also there are other vacancies for a System Administrator and a couple of developers into the bargain, all based in Oxford. The closing date on these is 27/01/08, so if you are reading this after then, sorry you are too late.

Prevention is better than cure

I loved this posting. I think it applies just as much to DBA’s and System Administrators. I have to confess I have had the occasional thrill of playing the hero at the scene of a car crash of a situation. It can feel good. In fact I think you can easily get addicted to being in this type of situation, where you hold the key piece of knowledge to fix a particular problem. I think I have at times been on the edge of the precipice of enjoying it too much and almost hoping for another opportunity to be a troubleshooter. But I see the danger of enjoying being a troubleshooter and I can see the benefit to the organisation of being a “troublepreventer”.

Recently, on the Oracle-L mailing list someone was asking for a list of tasks that a good dba should do on a regular basis. The person was given short-shrift in that they were told if there are tasks that your dba is performing on a daily basis, why are they doing them and why have they not automated them?

I also like the idea of “exception based reporting”, that is if everything about a system/database etc. is running within specified criteria, I don’t want to know about it, and only when a monitored criteria becomes outwith an acceptable criteria do you want to hear about it. However, this does require you monitor the system closely. As an example, I would suggest monitoring your I/O response time on your database, and/or critical query response time, and then if these change to be outwith acceptable times you flag it up and get someone to look at it.

Of course, I should eat my own dog food a bit more here.

How to be a failure

I was at a couple of London Business Forum events today, both featuring Steve Mcdermott. The first session was on Steve’s book, which is written as a series of steps NOT to take if you want to become a complete failure. I’m a sceptic by nature, I think most people with scientific training are. Though I hope I’m not a cynic, I suspect I do tend towards this. I started out by thinking steve was a bit of twat, but I really started to warm to him. He has had decades of presenting and really, really knows how to work a crowd; straight away he got on first name terms with a couple of members of the audience, and a few laughs really breaks the ice.

Steve highlighted a handful of his 44.5 steps to being a failure then expounded on them with copious examples and anecdotes. One of the first things he mentioned, which I took exception to (a little), was claiming that to fail you should never experiment and as an example of experimenting he gave 4 members of the audience various bells, rattles, and whistles with instructions to ring/blow/shake these after a set period of time. He claimed never to have done this before. I immediately thought about IT and that (obviously) you should always, always TEST any change before it is rolled into production. I think in the Oracle world there have been (well I guess probably for most of this decade anyway) a move away from magic bullets and changing/experimenting with parameters that claim to have magical properties in terms of tuning, to a far more scientific way of fixing your problems. Everyone knows Anjo Kolk, right?

His stuff on setting goals and trying to find someone to pay you for doing something you consider a hobby did get me thinking. I currently work for Nominet and I’ve been there 6 years, which in IT is a long time to be in the same place. I guess if you don’t have any goals you really are never going to achieve them.

I thoroughly recommend reading How to be a complete failure. It is very readable, filled with some great quotes and anecdotes, and hey, if you do some things he says don’t do you might not be so much of failure after all.

In the afternoon Steve was talking about “Presentation Mastery”. I must say straight away the guy knows how to communicate with an audience. I think what is not surprising, but is a fabulous technique, is how he does it really without any slides or notes to speak of. It is most definitely NOT death by Powerpoint. I took away a few key points, perhaps fairly obvious ones, but you have to know what you are talking about and have a story to tell. Have a dress rehearsal, oh and don’t just read what is on your slide! His idea of mindmapping your talk seems like a good one to me!

All in all a good day in the big smoke and only 2 hours travelling time Oxford to London door-to-door. I’d recommend checking future LBF events.