As I’ve been receiving questions regarding our assignment, I decided to arrange it in a hopefully-coherent FAQ. Please read on this before asking further questions.

Q: I am not quite understand what I am suppose to do
A: Read the problem again.Questions nr. 1 thru 4 are just there to aid you doing question nr 5.

Q: Can U give me a hint for nr.5?
A: Of course I can. Here they are:

1. First, create your database! I provided you with the database schema. All U have to do is to create the DDL (the “create table…” statements).
2. You can register the txt files and excel file as an ODBC datasource. How? See here.
3. As for the XML file, you can parse it using JAXP (Java API for XML processing). How? See here.
4. From your java program, connect to the ODBC datasources using JDBC. How? See here.
5. From your java program, connect to your MySQL database.
6. The rest is just plain querying the first data sources (ODBC) and inserting to the second (MySQL).

Q:Should I parse all the text files individually? (by Najihah Zamanhuri)
A: Well technically you can do that, but I’m sure you have better things to do with your time. I suggest you go for the ODBC way (see previous question). It is alot ‘cleaner’. And you can use your extra time for beautifying your program or saving rain forests.

Q: Will this assignment affecting our final mark?
A: Don’t know. The decission about this is on the lecturers, not tutors. But, I am required to report them. Dont’ worry, I will not waste any sincere effort. But instead of worrying ’bout final marks, how ’bout doing this for fun?

Q: I see other file types in the northwind folder. What should I do with them?
A: Just ignore them. You only need to load the data int txt,xls, and xml files.

Q:Should we submit hardcopies?
A: I’m not quite a fan for hardcopies. Just bring along your (softcopy of) java app, and DDL on the next session. Save rain forest by using less papers.

Q: I have other question. What should I do?
A: Drop a mail in my inbox : barkah[dot]yusuf[at]gmail[dot]com, Buzz my IM : barkahyusuf, or scout for my existence in postgraduate lab.

Q: I dont have MySQL.
A: Get one. It’s here and it’s free.

Q: I’d uninstall my java since I didn’t think i’d be needing it anymore.
A: Well then go get it again. It’s here

Q: What is the answer to life, the universe, and everything?
A: Forty two.

Have a nice day.

I just realized that you will require at least the database schema to create the DDL. Though, you can guess it from the data (as I did). Anyway, here is the schema: Northwind-db-schema (30KB)

I’ve written this crude tutorial to help students in DSS class work on their assignment. This step by step tutorial assumes you are using Microsoft Windows 2000.

• How to register Comma Separated Value (CSV) files as ODBC data source?

1. Go to ODBC Data sources :
2. Start > Settings > Control Panel > Administrative tools > Data sources (ODBC)
3. On the “User DSN” tab, click on Add button.
4. On the driver list, select the driver you will use. For plain text files, use “Microsoft text-driver (*.txt, *.csv)“.
5. In ODBC text setup dialog, fill in Data Source Name. This name will be used to refer this to this data source from inside applications.
6. Fill in description. And Uncheck “Use Current Directory”. The “Select Directory…” button will be enabled.
7. Click on “Select Directory…” button.
8. Browse to the directory where you put your text files, then Click OK. The directory selection dialog will close. This directory will be refered as DATA_DIR
9. Click on “Options>>” button. “ODBC text setup” dialog will be expanded.
10. Here, you can add or remove extensions to have the ODBC text driver regards or ignores certain file types under DATA_DIR.
11. If your plaintext files are delimited with other than comma (,) Click on “Define format…” button. Define text format will appear.
12. For every file listed in Tables list, change its Format to whatever formats your file has. Use custom delimited, and enter the separator character manually (for example, a semicolon ‘;’). Then click on Guess button to have the ODBC driver automatically guess what colums are there on the file.
13. Adjust other options accordingly.
14. Having done step nr 12 for each and every file listed, click on OK button. You will return to “ODBC text Setup” Dialog
15. Click OK on ODBC Text Setup Dialog.
16. Your data source is ready.
    

• How to register Microsoft Excel Spreadsheet (*.xls) files as ODBC data source?
  Same as previous tutorial, but in the driver selection dialog, select “Microsoft excel driver (*.xls)” instead.

• How to create a JDBC connection to a ODBC data source?
  Consider this code snippet below.

/* Load the driver class. */
Class.forName(“sun.jdbc.odbc.JdbcOdbcDriver”);

/* The URL for our JDBC connection. The format is "jdbc:odbc: */
String url = "jdbc:odbc:northwind";


/* Open connection */
Connection odbcConnection = DriverManager.getConnection(url);


/* Prepare a statement */
Statement stmt = odbcConnection.createStatement();

/* For example we will query customer.txt file */
ResultSet res = stmt.executeQuery("select * from customers.txt");

Note: As I have lost my thumbdrive recently, the following questions may not be exacly the same as you had on class. Here they are:

Do some small reseach to answer the following questions

1. Explain ETL phase in data mining?
2. What is XML? And what does XML serves in the ETL process?
3. Explain briefly what ODBC is?
4. Explain briefly what JDBC is? Stress on the difference and similarity between ODBC and JDBC!
5. You are required to write a small java application to perform ETL on the northwind database, to a MySQL database. To create the impression of the data coming from disparate sources, the data is stored in plaintext, MS-Excel Spreadsheet, and XML files.

Make sure you bring your java application and the data definition language (DDL) for your database for next session (March 20) when we’ll perform a few DSS tricks on the data.
As usual, questions goes to barkah[DOT]yusuf[AT]gmail[DOT]com.

ADDED: Apparently, This site forbids zip files to be uploaded. So I cannot upload the northwind data. However you can pick it up at Application Lab, it’s on the desktop of PC “10AppTeach07″ (the one just infront of teacher’s PC).

Relax mind

Relax body

Relax bowels

Relax.

Do not fall over.

You are a cloud.

You are raining.

Do not rain

While train

Is standing at a station.

Move with the wind.

Apologise where necessary.

Taken from the “Zen And The Art Of Going To The Lavatory“
by Grunthos the Flatulent, the poetmaster of the Azgoths of Kria, writers of the second worst poetry in the universe.

Appear in Douglas Adam’s, “The Hitchikers GUide To Galaxy“

Obrolan antara dua orang Indonesia, sebut saja nama mereka Markaban dan Rukmini, di suatu siang di kantin Halliburton, UTP :

markaban : Kalo udah lulus loe mo kerja dimane?
rukmini : Hmm, yang jelas ngga di Indonesia.
markaban :Knapa?

rukmini : Kalo pulang qualifikasi macam kita ngga akan dihargai. Kerja gajinya kecil.
markaban : Tapi dibutuhkan kan?
rukmini : (nada becanda) Iya sih. Lo sok nasionalis banget deh ngomongnya.
markaban : Elo ngomong seolah-olah nasionalisme itu kata-kata kotor.
rukmini : Nah elo sendiri, kalo udah lulus elo mo kerja dimana.
markaban : Di luar negeri juga.
rukmini : Nah tuh kan!. Sama aja elo sama gue.
markaban : Tapi alasan gue ngga sama kaya elo. Etlis, bukan alasan yg pertama.
rukmini : Jadi apa dong alasan elo?
markaban : Apa ya?…ini cuman conto: Dengan berada di luar, kita bisa jadi agen alih teknologi.
rukmini : Maksud loe?
markaban : Gue ngga mau muluk2 ngomong tentang berkontribusi untuk bangsa. Tau diri gue, emang gue siape?…
rukmini : Hmm…
markaban : Meskipun gue tau, kalo mo kontribusi ngga usah belajar dulu keluar (negeri). Jadi TKI aja.
rukmini : Tapi apa elo mau ngga jadi TKI?
markaban : Bukan gitu. Coba deh, apa TKI-TKI itu pernah kepikiran untuk “berkontribusi bagi bangsa dan negara”? Bahkan mereka juga ngga tau apa itu “makro ekonomi”. Tapi mereka nyata ngasi kontribusi. Ngga neko-neko.

Secara ekonomi makro, TKW Indonesia jelas kontribusinya positif thd PDB dibanding orang yang hobinya belanja keluar negeri.

rukmini : Hmmm….Do what you can, where you can?
markaban : Buat gue, niat berkontribusi ngga harus dibenturkan dengan kesejahteraan pribadi. Kadang gue sama sekali ngga liat itu sbg suatu yg luar biasa. Biasa aja.
rukmini : Tapi elo tetep kan ngga mau kerja di Indonesia?
markaban : Yah paling ngga kenapanya gue beda sama kenapanya elo. dan menurut gue itu bakalan bikin perbedaan nantinya….

…dialog berlanjut ke topik-topik lain yang ngga penting untuk diketahui orang lain.

During our first postgrad week, we had this so called ‘Research Methodology’ courses. The title to the show might be a bit dull for some. It is a traditional thing, throughout the world (or so they said), to start a postgraduate study with a session or two to welcome it’s newcomers to join the research community. More important than that, the session gives us vision to the true meaning of conducting this alleged ‘research’ thing.

I had to say, even with a label “Master by Research” attached to the program name (for PhD, AFAIK it is always ‘by research’), often I met with co-student here who did not realize what the phrase really means until he starts to find humps during the way. Me too was not an exception. I mean, I had an encounter with research. I’ve spent first 9-month after graduation, working for a research project, under the guidance of someone whom I consider a true researcher. He tough me the topsy-turvy road I will encounter during research. But still, I am encountering surprises here and there.

I mean, really, the activity of research, as I found it, really requires you to be able to manage your own agenda. Sets your own goal. Seeks your own questions. It is true that there will be associates to talk to. But in this state of exploration, sometimes you will find that you your self who knows your own problem and places to find the answers. Or sometimes the only person knows better than you live half way across the globe. Adding to the problem, your time will be limited. So this will sometimes become exceptionally depressing. I am not telling you about the so-called ‘pseudo-research’ where we can just follow definite instructions to come up with something already expected. Having to know this prospect beforehand really alleviates the shock (or at the very least ease it a bit).

The opening address by the program head went in breeze. The second session, “The Philosophy of Research”, was what arouses us. It was delivered with great enthusiasm by Prof. Mohd Halib, a 50 or so, hoary-Malaysian. He was, as he told us, a molecular biologist who turned in to a social scientist in his mid-career for a reason; he found that social science was more challenging as it deals with more parameters of human behavior. And he said that briskly in front of a room filled with students all majored in natural science or engineering. His style of pointing us in face and walked to and fro while, with a booming voice, emphasizing his points, really captivated us. Despite his enthusiastic upbringing, he was modest and down to earth. I have here a view quotes from him (with a slight adjustment, as my notes then weren’t pretty):

“…History has it that the inspiration to the laws of gravity was first struck a Malaysian. Alas, rather than an apple tree, the man was sitting under a durian tree. He didn’t survive to tell the world about the idea…”

“…Your duty here is not to dazzle the science community or the world. Your here to push the boundary of the body of knowledge you choose, just a tiny little bit. And when the time came, you get your degree and the world gets your contribution…”

“…The first problem you’ll encounter here is to find one…”

“…Someone’s authorship can never be freed from his ethical and moral standpoint…”

I can say that it was a fine start of our journey. And I am constantly hoping for the best here. While I am looking for some time to squeeze in a post or two, I am leaving with the professor’s slides attached, in hope that the wisdom will inspires me for days to come.