Multi-scale defines learning
https://rumble.com/vcexnm-multi-scale-defines-learning-part-1.html
https://rumble.com/vcexxw-multi-scale-defines-learning-part-2.html
Why have I been so focused on multi-scale data? Because learning can only occur when there are multiple scales. Why is that true, is sounds nonsensical, what is the relationship between the scale of a set of data and learning that data.
I need to better define multi-scale systems.
A multi-scale system has at least two levels. Each level has elements. Yet the elements on each level are different. How different? In what way are they different? Primarily the descriptive language of the elements is different. The elements on each level describe different attributes.
Since each level creates its own descriptive language, two different levels are not able to communicate directly with each other. Although that is a negative definition I like it.
The positive formulation might be, a higher level is a result of a non-linear function on (multiple elements from) the lower level (this starts to sound like Christian List)
Learning as opposed to memorisation requires this creation of a new descriptive space. This is Fernando Flores's definition of 'listening', the act of communication requires learning.
What first order and second order statistics do, such as mean and variance (and KMeans by association), is they attempt to represent the original data within the same descriptive space. This is a poor mans memory.
Learning creates a new descriptive space, an embedding, that is unique and distinct from the original descriptive space of the data. A model is created to map from the original descriptive space to the new descriptive space. The result of this mapping is a new representation that cannot coexist within the original framework.
Kohonen tried to preserve a relationship across scales with spatial feature maps, yet this undermines learning!
For example, cars have wheels and a chassis. The wheels are elements and the chassis is an element, are they are on the same level? Well since there is no relationship between them, they are independent, I would say they are not a multi-scale system. They are two independent systems.
However a car is a multi-scale systems, its higher level, 'car', is composed (a function) of at least two lower level elements, wheels and chassis.
Now we can measure the attributes of the lower level data:
a) wheels have a diameter and coefficient of friction.
b) a chassis has a passenger capacity
And the higher level data:
I) a car provides a timely service to a destination.
It is incorrect to formulate a sentence such as, the more passengers in a chassis the quicker the car reaches its destination. That is mixing levels. Or the greater a wheel diameter the quicker the car reaches its destination.
So how to create? How to create a higher level with its own new descriptive language?
What faddc measures is the relationship between centroids and not the relationship between data elements in the lower feature space (this I will prove later with categorical learning)
What that is akin to saying, when communicating between systems a new descriptive language is created.
The anchor to this new language is a shared experience. A set of two things that co-occur. This is the Semiotic definition of a 'sign'. When two symbols co-occur they are associated with each other and a new element (higher level element) is created, a sign. The neurobiological way of framing that statement is to say that when two neurons fire at the same time a connection is made between them. The statistical description might be that the distance between the two centroids is reduced if they co-occur, i.e. there is a correlation between them.
My algorithm measures the relative value of the 'Neurons' with each other. What is the distance between two neurons as opposed to the distance between two centroids? Centroids measure the distance in the data elements space as opposed to Neurons that measure distance in the Neuronal co-occurrence space. Distance is not measured in absolute terms, global objective metrics like in KMeans, but in relative space at the higher level. This creates a new descriptive space, the Neuronal space.
If we revisit the 'car' example. A car is a complex system, that provides a unique service that its parts cannot provide. With this new language I can communicate ideas that are unique, and can not be communicated at the lower level. The whole is greater than the sum of its parts, a car provides a language to describe concepts that are more than just wheels and chassis. The greater context, multiple people utilise cars, enables multiple instances of cars to be correlated and new conceptual ideas to emerge. Thus, within the higher level language of cars I can talk about traffic, multiple cars attempting to service a destination in a timely manner.
Comments
Post a Comment