Skip to content

Making AI Great Again

July 4, 2018

This is my own version of MAGA. To appear at the IJCCI 2018, Seville, Spain.


The Humanities of Maths/Computer Science

May 8, 2018

Last month I took the exercise of tracking the history of the term AI. In the process I found myself meandering towards some of the histories of maths that govern AI’s present state of the art.

Most scientists believe that mathematics is part of the Sciences. I do not hold such a view, rather, I do consider it really a part of Philosophy, so still part of Humanities. In going through the history of mathematical structures present in computer science, I am struck at the thought of how mathematics is a deeply human enterprise. I know this is obvious to some, and though this truth is somewhat present in a small way at the back of my mind, it is only now that I am dealing with it personally. This personal exercise brought this truth home to me in a profound way and I am delighted in that discovery.

I am amazed how the use of history can inform how one does his/her maths. In my examination, history can tell us what a mathematician or computer scientist went through. We can learn the trial and errors they went through and their character of tenacity in “keeping the faith”.  An example I can recommend worth studying is the struggle that Geoffrey Hinton went through on how he became the “grandfather of deep learning”. It was not only the story about the maths of neural network but also the story of how one’s passion and faith in an idea is benefitting society today.

Sigmoid in the ANN

March 28, 2018

Just playing around with two-layer Artificial Neural Networks (ANNs) wherein the hidden activation function is a sigmoid of the form found below

\sigma(a_{j}) = \frac {1}{1 + e^{-a_{j}}}

The final output will be of the form
a_{k}= \sum_{j=1}^{M}w_{kj}^{(2)}\sigma(a_{j}) + w_{k0}^{(2)}
where a_{j} is the result of the first layer and has the form
a_{j}= \sum_{i=1}^{D}w_{ji}^{(1)} + w_{j0}^{(1)}

We will now show that there is an equivalent network that computes the same thing but using hyperbolic tangent tanh.

\tanh(a) = \frac {e^{a}-e^{-a}}{e^{a} + e^{-a}} as the hidden function.
It is known that tanh(a) = 2 \sigma(2a) + 1 see here
tanh(a/2) = 2 \sigma(a) + 1 \Rightarrow \sigma(a) = \frac{1}{2} [tanh(\frac{a}{2}) - 1]
We now use this form of \sigma(a) in a_{k} and we can see that the same can be computed using tanh. Also, looking at the plot of these functions, it is not hard to see that they are a bijection of each other. QED \blacksquare

Note: Notations and idea inspired by C. Bishop. Pattern Recognition and Machine Learning, 2006

When “AI” is “real” AI

March 15, 2018

I remember 20+ years ago. Twenty years ago, if you ever told anyone in IT, especially those in IT management that you were working in the field of AI, you would be given a skeptical look. They would not want to know you or what you do. Why? Because AI, 20 years ago was a failed promise. I remember an agent-oriented company in the USA even shied away from ever mentioning on their company website that they were using agent technology in their product, despite being a thoroughgoing supplier of an agent platform! AI used to be an irrelevant idea.

So after working for more than 20 years doing agent-related research, I sit in wonder and ask, what happened? Now, the AI label is being thrown around everywhere. Suddenly it is in vogue. People love it and praising it when 20 years ago, it was despised. What changed? Until you take a closer look…

When people talk about a product having “AI”, what they mean is that they have a product that can predict. Prediction is indeed a part of AI, but that is a very small part of it, it is not the sum of it all. In fact, it is not even the crucial part of it.

Will the real “AI” please stand up?

In the wider, classically and often cited AI book, the book of Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, lists 4 approaches to AI:

  • Acting Humanly
  • Thinking Humanly
  • Thinking Rationally
  • Acting Rationally

By the Russell-Norvig definition, something has AI if that thing can operate autonomously, perceive their environment, persist over a prolonged period of time, adapt to change, and create and pursue goals. In other words, Russell and Norvig, go for a rational agent. A rational agent acts in such a way that it achieves the best outcome or the best-expected outcome based on its goals. The ability to predict is a component of this rationality, definitely, but it is not the sum total of the agent’s being. By this definition then, having the ability to predict is not necessarily AI. Unless that predictive power lies inside an agent, then, only then we may say, there is AI there.






R Closures and λ-calculus

September 12, 2016

What I like about R is that it takes its inspiration from Scheme. Being a functional programming language, R indeed supports the idea of closures. What are they? This is my attempt to unravel the mystery about closures.

First what closures are not. Some writers have confused this notion with that of algebraic closures. Functional programming closures have nothing to do with algebraic closures. Don’t you believe it if  you get told it came from Group Theory. It does not. An algebraic closure is a property of an algebraic structure. Namely if a, b \in G and \bullet is an operation in G, then a \bullet b \in G. In other words the result of applying the operation on the members of G, is also a member in G. This says that G is closed under the operation \bullet, whatever that operation may be.

Functional programming closures have something to do with how the language should handle free variables in relation to functions. Since functional programming takes its cue from lambda calculus, let us define what free variables are in this  computational system. It is best done by an example below:

Consider \lambda x.yx . In this expression, y is free as it is not controlled by the lambda symbol, but x is bound because \lambda is enclosing that x, as we read from left to right. This is where the functional programming (FP) closures come from. FP closures tell us how it will handle those free variables when they occur in a function definition. The way this is explained in FP is in the area of environments and R have those too. Simply put when a function is defined, the system remembers that state of the environment where and when that function is defined. In other words, it remembers the declarations made inside the function such as names of variables and nested functions declared inside it.

The best way I think this is illustrated is in the way FP closures mimic classes in OOP languages. Look at this code.

make_bal<- function (){
  val <- 0
  bal <- function (method){
    add_method <- function(x){
      val <<- val + x
    get_method <- function() {
    if (method == "dep") {
    else {

In R, the label “function” acts as our lambda declaration. In line 2, val is a free variable because it is not defined as an argument of the function. In line 4, x is bound. It is defined as a parameter in the function definition of add_method. However val is not accessible outside make_bal because it is defined inside it and so we could say it is a  “closed” to the outside world. In line 5 we use the <<- operator similar to set! in Scheme. That operator will hunt for val up the environment chain and will get the first one it finds which is in line 2.

So let us ask R to load our make_bal.

> account<-make_bal() 
> account("get")()
[1] 0
> account("dep")(50)
> account("get")()
[1] 50
> account("dep")(50)
> account("get")()
[1] 100
> withdraw<-account("dep") 
> withdraw(-50)
> account("get")()
[1] 50

In 1, we define a function from it, and call it account. In 2, we check that val is starting at 0. In 4, we made a deposit of 50. Each time we make a deposit, our balance increases as seen in line 8. Finally we can define a function called withdraw provided the amount we pass is always negative and as can be seen when we call it in line 13, the balance has been reduced by the amount we took out of our balance.

One can see that in an FP like R, the notion of classes will not be missed because closures handle that need.

As usually happens in computer science, computer scientists borrow notions from mathematics and this use of the term “closures” is one of them. Then also as it usually happens, it produces negative consequences, rather than clarify sometimes it confuses people in the process. It would have been better had computer scientists chosen a different term.

Credits: Andy Balaam’s Scheme Closure.

Calling Excel from R in a Mac OS X Environment

July 31, 2016

I aRlogom one of those who often need to flip between Excel or R depending on the needs of the client. Calling Excel in R is not that trivial so here are some notes on how to do that. So let’s say that you have an Excel VBA program that you want to take over after R has done its job.
You use the system2 command in R like so
> system2(“/Applications/Microsoft\\ Excel”, c(“/Users/Extranosky/Temp/VehicleRepair.xlsm”))
The second argument should not have spaces in its file or directory name. I have not had success in passing a file name to be opened by Excel in this platform that had spaces in its directory.

This command will call up Excel open up the file as an argument and hopefully you got an open event in that VBA so things can be done seamlessly and automatically.

An example of how math messes up your mind

October 28, 2015
Symbols of infinity from Wikipaedia

Symbols of infinity from Wikipaedia

In this post I will show an example of how mathematics can be very mysterious. I showed this to what might be classified as “first year in maths” students and they came out of class perplexed, as if their mind had to hit the reset button and reboot.

Theorem: Let \mathbf{N} be the set of natural numbers and \mathbf{Z} the set of integers. We have \mathbf{N} \subseteq \mathbf{Z} and |\mathbf{N}| = |\mathbf{Z}|.

This means that though \mathbf{N} is a subset of \mathbf{Z}, the size of \mathbf{N} is equal to the size of \mathbf{Z}. In technical language the cardinality of the natural numbers is equal to the cardinality of the integers, What duh?


a.) \mathbf{N} \subseteq \mathbf{Z}, this is trivial because the natural numbers \mathbf{N} are just the positive integers in \mathbf{Z}. So every element of \mathbf{N} is found in \mathbf{Z}.

b.) In order to show that the size of the two sets are equal we need to establish a bijective function from one set to the other. That is a function which is both surjective and injective. Another way of saying this is to say that we need a function that is onto and at the same time one-to-one from one set to the other. Obtaining such a function proves that the size of both sets are the same.

We will just get the one suggested by wikipaedia: We let
f : \mathbf{Z} \rightarrow \mathbf{N} with f(x) = 2|x| when and only when x \in \mathbf{Z} and x< 0,

else f(x) = 2x + 1 when and only when x \geq 0.

1.) f is one-to-one, i.e. injective. Let f(x)= f(y), then we have two cases, either 2|x| = 2|y| or 2x+1 = 2y+1. The first case we have \Rightarrow |x| = |y|, \Rightarrow x,y < 0, \Rightarrow x = y. On the other hand, if the second is the case then again this \Rightarrow 2x+1=2y+1 \Rightarrow x = y.

2.) f is onto, i.e. surjective. Let b \in \mathbf{N} then b is positive and either even or odd. If even and positive, then in general \Rightarrow  b = 2|a| for some integer a,(the property of even numbers). Since a is an integer, then it is an integer in \mathbf{Z} and |a| = b/2 so that f(a) = 2(b/2)=b and choose a < 0. On the other hand, if b is positive and odd, then b = 2a + 1 for some integer a (property of odd numbers) \Rightarrow a = (b - 1)/2 and a has to be positive since b is positive, i.e., a \geq 0 and so we have f(a) = 2(b -1)/2+ 1 = b. So in both cases we have seen that for every b \in \mathbf{N} , we have found a matching a \in \mathbf{Z}. \blacksquare

How can a subset have the same size as it's superset? If this does not boggle your mind, perhaps you missed the point. The reason for this is that we have here two infinite sets, and this mystery only happens when infinity is involved. Now some mathematicians are not happy with this that is why they do not believe in infinite sets. It seems infinity is just a concept that has no matching physical reality and we can be indifferent with it. I suggest the concept of infinity is a metaphysical concept. So can one can reason that it is a concept that exists just in our mind and is not "real". Just like unicorns or fairies? I do not think so. There is no reason for us to believe in unicorns and in fact not all civilisations believe in the mythical horse. However the concept of infinity is different. It is because the concept per se is a necessity. The mind requires it when presented with the nature of numbers. For is it not true that the set of natural numbers is infinite? We can conceive it and by force of nature admit it. It is a necessary truth so in that sense infinite sets are real and transcends material physicality.