Connecting to Oracle 12g with the Instant Client and Excel VBA

Excel References

One rather old application I’ve supported for several years loads data from Excel spreadsheets into a reporting database. These Excel spreadsheets have always been manually updated by several users. However, because the data that the users are entering into these spreadsheets are first being entered into another separate application database, these users have been doing double-entry – a redundant process which can be easily remedied by various means.

Ideally, the solution for this problem would be to extract the data from the application database and load it into the reporting database using an SSIS package. Unfortunately, that would require some redevelopment of the application which loads data into the reporting database, and we (and the customers) have no bandwidth for that. So I came up with a quick workaround that made everyone happy – using a VBA macro to automatically populate the spreadsheets with data when the users open them.

The tricky part here was getting Excel to connect to Oracle with the least amount of work having to be done on the users’ PCs, which are remote from my location.

First of all, since these users don’t require SQL Plus or any development tools for Oracle, the full client software was unnecessary. Also, the Oracle Instant Client could be pushed to the users with Altiris Deployment Solution.

I had the Instant Client software installed on the PCs, then I added the requisite database connection information to the tnsnames.ora file.

Nota bene: In the Instant Client (or at least in our setup, using version 11.2.0.4) the tnsnames file is in
C:\oracle\instantclient_11_2_0_4 rather than in C:\oracle\product\11.2.0.4\client_1\NETWORK\ADMIN as it often would be in the full Oracle client.

The connection in VBA was simple enough, but not immediately obvious – notice that the connection string includes “Microsoft ODBC Driver for Oracle” rather than an Oracle driver; even though this is used, no ODBC connection needs to be set up in the ODBC Data Source Administrator. It is only imperative that the proper entries exist in the tnsnames.ora file, and that the Microsoft ActiveX Data Object Library is installed and referenced in Excel. (Add References by navigating to Tools –> References in the VBA editor in Excel.)

Excel References

In a subroutine, this code was used to connect to the database and pull data.

    Dim SQL_String As String
    Dim dbConnectStr As String
    Dim con As New ADODB.Connection
    Dim recset As New ADODB.Recordset
    Dim strUid As String
    Dim strPwd As String
    Dim strEnv As String
    Dim strDSN As String
    Dim iRow As Integer    
    

    strEnv = "prod"
    strUid = "username"
    strPwd = "password"

    If strEnv = "prod" Then
        strDSN = "(prod database net_service_name* from tnsnames)"
    Else
        strDSN = "(dev database net_service_name* from tnsnames)"
    End If
          
    dbConnectStr = "Driver={Microsoft ODBC for Oracle}; " & _
            "Server=" & strDSN & ";" & _
            "uid=" & strUid & ";pwd=" & strPwd & ";"
       
    con.ConnectionString = dbConnectStr    
    con.Open   
    
    SQL_String = "(insert SQL query here)"
           
    recset.Open SQL_String, con

    iRow = 0 
    Do While Not recset.EOF
         'Have a loop here to go through all the fields
        Sheet1.Range("A" & iRow).Value = recset.Fields("colname") ' colname = Column Name from SQL query
        
        ' &c. ...

        iRow = iRow + 1
        recset.MoveNext
    Loop

    recset.Close
    con.Close

* net_service_name

Oracle Database Error ORA-01013 in Excel VBA After Migrating Application to New Server

VB run-time error ORA-01013

I was testing an old Excel application that I had moved from one server to another and kept getting this error when running it on the new server:

VBA run-time error ORA-01013

The error ORA-01013 (“user requested cancel of current operation”) is commonly a timeout issue. What could be causing a query to timeout on one server, but not another?

In debugging the code, I found that no CommandTimeout property had been set on the Connection object.

Dim cnnRawData As ADODB.Connection
'...    
Set cnnRawData = New ADODB.Connection
cnnRawData.ConnectionString = glbConnectString
cnnRawData.Open
cnnRawData.Execute glbSetCurrentSchema

If it is not explicitly set, this property will default to 30 seconds. However, this did not explain why the query did not timeout on the old server.

The connection to the Oracle Database was being made via ODBC, so I opened the 32-bit ODBC Administrator to see if the connections had been set up differently. This is where I realized what the problem was.

On the old server, the connection configuration looked like this:

ODBC config on old server

And on the new server it looked like this:

ODBC config on new server

See the difference? (Aside from the Description field, that is!)

It’s the “Enable Query Timeout” checkbox in the Application tab. By default, when creating a new connection, it is checked. I unchecked the box and saved the connection, and the error went away.

This is not necessarily the best way to do it, however. I might want some queries to timeout, and not others. In this case the solution would be to set the CommandTimeout property for this particular VBA macro to 0:

Dim cnnRawData As ADODB.Connection
'...    
Set cnnRawData = New ADODB.Connection
cnnRawData.ConnectionString = glbConnectString
cnnRawData.CommandTimeout = 0
cnnRawData.Open
cnnRawData.Execute glbSetCurrentSchema

This would prevent timeouts on this particular connection, but not any others that might be made using the same ODBC connection.

Errors in Pasting Data into Datasheet View in SharePoint 2010

As a part of my project to convert a Lotus Notes system to something else, I am making good use of SharePoint Lists to replicate the functionality of some of the less complex Notes applications.

For some of these databases, I am exporting the documents into Structured Text, then converting the structured text to Excel spreadsheets.

A particularly troublesome error occurred when I was trying to copy a large number of rows from an XLSX file into the Datasheet View of a SharePoint 2010 List. I had originally copied several thousand rows into the List, only to realize that I had accidentally selected the wrong range of columns (which made for some interesting looking data when the data types were dissimilar). To fix this problem, I deleted the rows from the Datasheet View, and then attempted to copy-and-paste them again – this time using the correct range.

At this point, SharePoint gave me an several different errors indicating that an incorrect number of columns had been selected, that some cells were read-only, and that pasted data would go off the sheet.

After verifying multiple times that I had indeed selected the right number of columns and the correct columns in the Excel file, I finally decided to reload the Datasheet View in the browser with F5.

Voila, no more error when trying to paste the data. I do not know if this is a “known issue” or not, but it seems that reloading the Web page after deleting data from the Datasheet View is necessary prior to attempting to load new data.

I did find a good site for tips about copying from Excel into a Datasheet View.

Exporting Data from Lotus Notes Databases

Exporting from Lotus Notes

Lotus Notes was innovative and, by most accounts, a great collaboration software in its heyday. Though there are still some 35 million users of Notes, there are regular rumors of its imminent demise.

Because of this, some organizations that have used Notes over the years may want to move to some other system, such as SharePoint. Getting data out of Notes for reporting or for use in a different system is not very straightforward, as there are several options depending on your needs.

The first is to connect Notes to SQL, which I covered back in June 2012.

Exporting from Lotus Notes
Exporting from Lotus Notes

Another is to export the data into Lotus 1-2-3, and then into Microsoft Excel, if need be. This is problematic for many Excel users today, as you cannot open Lotus 1-2-3 files with versions of Excel newer than Excel 2003.

Exporting view data into a comma separated value (.csv) file is another option. I have read that this particular option does not always work exactly right, especially if there are commas or some other special characters in the data, so YMMV.

The main problem with all of the above options is that you can only export data shown in a particular view with these methods. Unless you want to create (or already have) a view that has all document fields in the view, you won’t be able to see them all. If you want to export all the data in each document, you must export documents into what is called “Structured Text”.

Structured Text, while readable by text editors, cannot be directly imported into CSV or other formats using only Word or Excel. However, some tools been developed for this purpose. One example of such a tool is the Structured Text Parser (STP) written by James J. Schwaller. Tools such as these will make converting data from Lotus Notes into other formats or for use in other systems much simpler, and will not require hiring contractors or buying expensive tools that you will not need once your data is converted.

Good luck!