I wouldn't claim it's unknowable. Just that there's little evidence so far to suggest any form of sentience could arise from current machine learning models.

That hypothesis is not verifiable at present as we don't know the ins and outs of how consciousness arises.

Then it would logically follow that all the other functions of a human brain are similarly "possible" if we train it right and add enough computing power and memory. Without ever knowing the secrets of the human brain. I'd expect the truth somewhere in the middle of those two perspectives.

Lots of things are possible, we use the scientific method to test them not speculative logical arguments.

Functions of the brain

These would need to be defined.

But that means it should also be reproducible by similar means.

Can't be sure of this... For example, what if quantum interactions are involved in brain activity? How does the grey matter in the brain affect the functioning of neurons? How do the heart/gut affect things? Do cells which aren't neurons provide any input? Does some aspect of consciousness arise from the very material the brain is made of?

As far as I know all the above are open questions and I'm sure there are many more. But the point is we can't suggest there is actually rudimentary consciousness in neural networks until we have pinned it down in living things first.