But no support for was provided. — L'éléphant

Reference? — jgill

https://youtu.be/sMb00lz-IfE?si=7hnqXoiyVOdTHOJL

I don't know about this OP. It is uncharacteristic of Benj96 topics. — L'éléphant

Interesting. What is characteristic of my topics? I'll admit perhaps I jumped the gun on this one but I was captivated by veritasiums video on the notion and wished to share it here.

Please see reference link

In a statistical mechanics book, it is stated that "randomness and information are essentially the same thing," which results from the fact that a random process requires high information. . . . .
But, later it says that entropy and information are inversely related since disorder decreases with knowledge. But, this does not make sense to me. I always thought that entropy and randomness in a system were the same thing. — Gnomon

Perhaps "the randomness and information are essentially the same thing" simply means that you cannot compress something that is random or you will lose information (about the random sequence). At least that is the way I understand it.

If you have a random sequence, the only accurate way to model it perfectly is by writing the sequence in it's entirety. If you have a sequence that isn't random, let's say Tolstoi's War and Peace. I can refer to it and not copy paste here the whole book. An accurate shorter version is just to refer what book, when printed and perhaps which translation in what language from Russia.

The disorder decreasing with knowledge is quite another issue.

Perhaps "the randomness and information are essentially the same thing" simply means that you cannot compress something that is random or you will lose information (about the random sequence). At least that is the way I understand it. — ssu

You can't compress a random sequence of characters or a random collection of objects, you can only describe it, and that description will be 1:1. Whereas as soon as it is ordered, that order can be leveraged to create meta-data about the object.

I don't know if I agree with Verisatium's reasoning in this regard (that's the video that is referred to above which was the source for this thread) - chaos doesn't contain or convey information of any kind. It can't be compressed but how is that a criterion for 'information-bearing'? At 3:17 where he says that a completely compressed file is completely random - not sure about that, either. Otherwise, how could it be de-compressed, or intrepreted, at the receiving end? If it were totally random, then there'd be nothing to interpret. So I'm still not sold on the 'information=entropy' equation.

But I like that he recognises that quantum physics undermines LaPlace's daemon. Kudos for that.

Interesting. What is characteristic of my topics? I'll admit perhaps I jumped the gun on this one but I was captivated by veritasiums video on the notion and wished to share it here.

Please see reference link — Benj96

Sorry, I tried watching it. But the minute I heard the word "random" I lost interest. They were talking about examples such as the sun rising. Randomness is not the opposite of atmospheric stability or climate stability.

I think as a community of philosophy, we've become lazy and throw here and there clichés like "randomness" and "predetermined".

As to the characteristic of your OP -- normally they were thoughtful. Not this one.

You can't compress a random sequence of characters or a random collection of objects, you can only describe it, and that description will be 1:1. — Wayfarer

That's what I tried to say, but that's a better way saying it. And the statement describing "information" is basically about this inability of compression. I guess.

- chaos doesn't contain or convey information of any kind. — Wayfarer

I think the problem is that the meaning of "information" here is quite specific and doesn't relate to what we usually think of "information". Perhaps using the term "raw data" would be more appropriate. Data refers to "things known or assumed as facts, making the basis of reasoning or calculation", so that isn't helpful either. I think people would understand the difference between "information" and "data" better.

And if someone thinks that a random sequence if continued to infinity has the all the information in the World, shows just how difficult it is for us to grasp infinity.

They were talking about examples such as the sun rising. Randomness is not the opposite of atmospheric stability or climate stability. — L'éléphant

The sun rising is not an atmospheric stability nor climate stability phenomenon. Let's not conflate the cosmological with local planetary climate trends.

I don't know if I agree with Verisatium's reasoning in this regard (that's the video that is referred to above which was the source for this thread) - chaos doesn't contain or convey information of any kind. It can't be compressed but how is that a criterion for 'information-bearing'? At 3:17 where he says that a completely compressed file is completely random - not sure about that, either. Otherwise, how could it be de-compressed, or intrepreted, at the receiving end? If it were totally random, then there'd be nothing to interpret. So I'm still not sold on the 'information=entropy' equation.

But I like that he recognises that quantum physics undermines LaPlace's daemon. Kudos for that. — Wayfarer

I'm inclined to agree with you. I don't see how a compressed file can be both random and decryptable. Something random would not be informative at all. Perhaps I was mislead by some parts of the video

At 3:17 where he says that a completely compressed file is completely random - not sure about that, either. — Wayfarer

I don't see how a compressed file can be both random and decryptable — Benj96

Well compressing a file is eliminating all the redundancies or regularities in the data. So if you keep compressing data you are removing all the patterns in it. Like you said, random data cannot be compressed. Why? Has no regularities or redundancies in it. If it did, then it follows you can make further compressions.

Right. But the error I think the Verasatium presentation makes is then to equate non-compressibility with information - that a completely random string carries the greatest amount of information, because it can't be compressed. Whereas I think a random string embodies no information whatever.

That said, I frequently watch that Youtube channel, he's a very good popular science commentator. But I think this was not one of his better efforts.

The sun rising is not an atmospheric stability nor climate stability phenomenon. Let's not conflate the cosmological with local planetary climate trends. — Benj96

Again you missed.

I said, "examples such as". My next statement is a generalization of the way that videos like this develops. I'll explain to you the misuse of randomness: We argue in favor of randomness whenever there is a phenomena that goes out of order, unexpected, or out of place. Randomness has become the crutch for anything that falls outside of intelligibility. Instead of just saying unintelligible, we say random. You know why we say it? Because we don't like to think that everything has an explanation, that everything has an origin. Ultimately we don't like to think that there's always something, rather than nothing. There was never a point in the universe, that the nothing existed. This is what is hard to comprehend.

Here is a short passage:

How does the atmosphere rotate with the Earth?

Robert Matthews

Asked by: Rod Lennox, Colchester

Bound to the Earth by gravity, most of the atmosphere spins along with it as a result of friction with the ground and the viscosity or ‘stickiness’ of the different layers of air above it.

Above 200km, however, the incredibly thin atmosphere actually spins faster than the Earth. The cause of this bizarre ‘super-rotation’ effect remains unclear, but has also been detected on Venus.

But the error I think the Verasatium presentation makes is then to equate non-compressibility with information - that a completely random string carries the greatest amount of information, because it can't be compressed. Whereas I think a random string embodies no information whatever. — Wayfarer

Yeah, I guess that is fair. I haven't watched the video so I can't comment too much. Maybe it again comes down to this whole use of the word 'information' being ambiguous again. In one sense, when talking about code, you could argue all the semantic information is with the whoever is coding and de-coding, and the information they both have or don't have shapes the kind of messages they are required to send to each other. The information isn't in the code itself.

"The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages. The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design."

Shannon's information theory was I think intended to be about formal constraints on representing variables that produce distinct events probabilistically. (Edit: but the information being represented is with those using the code, not strictly in the code).

There was never a point in the universe, that the nothing existed. This is what is hard to comprehend. — L'éléphant

I never believed there was such a point in the universe when nothing existed. I dont find that hard to comprehend. My focus on randomess is not contignent on that.

Not really sure what it has to do with randomness as a phenomenon. If randomness is born from the very fundamentals of physics (which quantum physics seems to suggest), then even if everything from that point onwards is deterministic, explicable and predictable, the underlying origin is still random and unpredictable.

In that case randomness would appear to trump the determined and explicable, the patterned. If we cannot know exactly where particles will appear or annihilate but only give a statistical wave function of the distribution of possible locations, that would entail a trickle up effect of integral chaos within the system.

I don't believe all information in the universe is predictable because of heisenbergs uncertainty principle. Sure 99% of things can be non random but even if the fundamental 1% is that throws a huge spanner in the works

Yeah, I guess that is fair. I haven't watched the video so I can't comment too much. Maybe it again comes down to this whole use of the word 'information' being ambiguous again. — Apustimelogist

The definition of randomness as incompressibility is due to Kolmogorov.

https://en.wikipedia.org/wiki/Kolmogorov_complexity

There are other definitions of randomness in various other fields.

The rationale of Kolmogorov's idea is that, for example, there is absolutely nothing random about the digits of pi. They're the deterministic output of any number of closed-form expressions that can be programmed on a computer. The same goes for every other familiar irrational constant like sqrt(2), $e$, the golden ratio, and so forth. These are all computable numbers. Their infinitely-long digit sequence can be deterministically cranked out by a computer program or mathematical expression having relatively few characters.

Whether this satisfies or offends anyone's individual philosophical sensibilities is a matter of personal preference.

Yes, but then it is another issue how that might relate to what people call information.

Yes, but then it is another issue how that might relate to what people call information. — Apustimelogist

There are other definitions of the word, as I indicated in my post. Back in the day you could dial 411 and a nice lady would come on the phone and say, "Information." Alas there are no more dials.

I indicated in my post. — fishfry

That would suggest you are implying information is randomness; the original point of my post presupposes this is not necessarily the case.

That would suggest you are implying information is randomness; the original point of my post presupposes this is not necessarily the case. — Apustimelogist

I'm not Kolmogorov. I only identified him as the originator of the idea that randomness is measure by the degree of incompressibility. I said in my post, "There are other definitions of randomness in various other fields."

I'm having trouble following your posts. I gave Kolmogorov's definition then said there are other definitions. I have no disagreement with anything you wrote, but I don't see how it bears on what I wrote.

As I said, the point of K's idea is that when we give a finite-length description of the digits of pi, as we can easily do, we are showing that pi only encodes a finite amount of information. That's Kolmogorov's idea.

Suppose I want to transmit the sequence of the decimal digits of pi to my friend via telegraph, back in the day when telegrams were expensive. (Are telegrams still expensive? Do they still exist? I have no idea). I could sent each of the infinitely many digits. Or, I could just sent one of the closed-form, finite-length expressions for pi. Much cheaper. That's compressibility. It shows that the digits aren't random at all, but strictly deterministic.

Now, I can see how one would object to saying that shows pi doesn't contain much information. But it doesn't! I can express all of its infinitely many digits with a short, finite-length string of symbols. That takes very little information.

I am not planting a flag and saying this is the only definition of information. I'm just trying to motivate Kolmogorov's idea. And failing badly, apparently.

I'm having trouble following your posts. — fishfry

I just don't understand what the intention of your initial comment was. From my perspective it doesn't follow from the rest of the thread I was following.

I just don't understand what the intention of your initial comment was. From my perspective it doesn't follow from the rest of the thread I was following. — Apustimelogist

Oh I see. I can explain what I had in mind.

The OP was inviting discussion on the intuitively confusing idea that a random sequence of symbols contains the most "information." Many people think that if you're transmitting information, it's NOT random. It's noise. So the compressibility argument seems to be saying that there's more information in the noise than in the signal. So it's a seeming paradox.

Then the thread as I understood it, though I was not reading closely, became a discussion of how sensible a definition of randomness that was. I gather this had been exposited by 3Blue1Brown Youtube channel, and some people were even questioning the video. I'm repeating all this from a very shallow glance at the sense of the thread over the last few days.

Hold that thought and let me change the subject for a moment. In another thread, a paradoxical puzzle was presented that seemed to violate our intuitions about how probability works when randomly choosing a natural number.

I happened to know the puzzle could be explained through an understanding of the axioms of probability devised by the Russian mathematician Andrey Kolmogorov in 1933. So I wrote that up over in that thread.

Then I remembered that it was the very same Andrey Kolmogorov who had come up with the idea that you could measure the randomness of a string of symbols by its incompressibility. And that this was a measure of information. Kolmogorov is both the probability guy and the incompressibility guy.

I said to myself, "In that other thread [meaning this one], people are discussing the compressibility idea, and whether it makes sense. I thought pointing the thread to the originator of the idea might help add context.

So I popped in over here to toss in a little factual nugget that was not intended to change anyone's mind about anything. I was not referencing anything in the thread. I only wanted to let people know where the compressibility idea came from, so they could place it in context in the study of information.

Just intending to add a factoid, not making any argument at all.

I can see how I could have contextualized my remark better.

I gather this had been exposited by 3Blue1Brown Youtube channel, — fishfry

It's from a channel now called Verisatium, although under an earlier name. The original video wasn't cited until the top of page 2, you can review it here https://www.youtube.com/watch?v=sMb00lz-IfE . (I've viewed quite a few Verasatium presentations and overall found them pretty good, but I'm very dubious about some of the claims in this one.) I can definitely see the relevance of the Kolmogorov complexity idea, the video would have been better if it had been informed by it.

...nobody really knows what entropy really is so he will have the advantage in winning any arguments that might occur... — Wayfarer

This provides a good representation of this thread. Use a term with sufficient ambiguity, "information" in this case, so that the unintelligible is adequately hidden within what is proposed as intelligible, and it will appear like you are saying something intelligent. The majority of people are fooled by appearances.

And has been pointed out, ‘information’ is the logical master metaphor for our day, as ‘mechanism’ was previously.

It's from a channel now called Verisatium, although under an earlier name. The original video wasn't cited until the top of page 2, you can review it here https://www.youtube.com/watch?v=sMb00lz-IfE . (I've viewed quite a few Verasatium presentations and overall found them pretty good, but I'm very dubious about some of the claims in this one.) I can definitely see the relevance of the Kolmogorov complexity idea, the video would have been better if it had been informed by it. — Wayfarer

Veritasium, right. Thanks for the correction. I like Veritasium but didn't watch this video. He has some off days as we all do. If he mentioned compressibility but didn't credit Kolmogorov, I can see why that would lead people to think he was expressing a fringe idea, or whatever the criticism of the video was.

ps -- Just watched half the vid. Hopelessly muddled, definitely a misfire for Veritasium. I gave up when he started trying to claim that quantum mechanics is the cause of increasing information. This video was totally confusing. I can see why it sent the thread down a rabbit hole.

the unintelligible is adequately hidden within what is proposed as intelligible, and it will appear like you are saying something intelligent. — Metaphysician Undercover

You just described your own posting style.

You just described your own posting style. — fishfry

That's very observant of you fishfry. Generally speaking, one's language use is a reflection of the conventions and habits which the society has immersed that person in. I like to seek, determine, and then exaggerate within my own usage, the various ambiguities, misleading implications, false representations, and overall misgivings of deceptive habits and conventions which permeate our communications, thereby laying them bare, exposed for the world to see, so that perhaps, at some point in time, the general population will start to realize that something needs to be done about this situation. Wayfarer knows me as the obfuscator.

That's fair. I didn't actually think you were making an argument, I just didn't know where you were coming from. To me it looked like you had misunderstood the intention of my quote.

That's very observant of you fishfry. Generally speaking, one's language use is a reflection of the conventions and habits which the society has immersed that person in. I like to seek, determine, and then exaggerate within my own usage, the various ambiguities, misleading implications, false representations, and overall misgivings of deceptive habits and conventions which permeate our communications, thereby laying them bare, exposed for the world to see, so that perhaps, at some point in time, the general population will start to realize that something needs to be done about this situation. — Metaphysician Undercover

Your thesis is that someday, Internet archeologists are going to discover this thread and go, "My God, math is wrong!"

Wayfarer knows me as the obfuscator. — Metaphysician Undercover

Wish I'd figured that out long ago.

Your thesis is that someday, Internet archeologists are going to discover this thread and go, "My God, math is wrong!" — fishfry

It's actually an ongoing process, the evolution of thought. Look at Russel's paradox for example.

It's actually an ongoing process, the evolution of thought. Look at Russel's paradox for example. — Metaphysician Undercover

In the evolution of thought, people are going to decide math is wrong because it doesn't actually refer to anything? I thought that was a feature.

I see what's going on with the Lounge. All the political posts, Trump and Ukraine and Gaza, got moved over there.

What kind of philosophy considers only lofty, abstract issues, and turns its face from matters of life and death in the world today? Of course those topics generate heat. From what I can see, the Lounge is now the best part of this site.

Have we hopelessly hijacked this thread? What happened to information and randomness?

In the evolution of thought, people are going to decide math is wrong because it doesn't actually refer to anything? I thought that was a feature. — fishfry

In practise the math always refers to something. In theory it is designed to be applicable to a very wide range of circumstances. When the theory is not being used to refer to anything, it sits idle. So it only "doesn't actually refer to anything" if it never gets used, in reality it refers an endless number of things.

From what I can see, the Lounge is now the best part of this site. — fishfry

It is a pretty cool place to hang out, lots of activity down there. And they call it The Lounge. Now that's a misnomer.